NCEP Central Operations (NCO) continued to migrate NAWIPS functionality into the AWIPS II system during the month of July, according to NCO Chief of Systems Integration Michelle Mainelli. "NCO also spent time testing the automated data flow from the Central Computing System to the AWIPS II Test Bed located in the World Weather Building," says Mainelli. Test data includes NCO-provided model data and observational data types including radar mosaics and NCEP-specific satellite imagery.

"The focus of our software migration and automated data flow this past month was to prepare for Phase Two of our Forecaster Integration Testing (FIT), which is expected to commence on 15 August," Mainelli continues. "The purpose of the FIT is to test the functionality of the National Center Perspective. Forecasters and IT Specialists from NCEP Centers will go through their pre-determined test plans of creating products, retesting fixed Discrepancy Reports from the February FIT, and conduct initial performance testing."

On July 26, 2011, David Plummer, the NCO Systems Integration Branch AWIPS Team Lead, presented the third AWIPS II training module webinar, which focuses on the National Center Perspective, introducing the user interface and variance training. Unidata will provide links to the training material when it becomes available.

Unidata Preview of AWIPS II

Participants in the GEMPAK portion of the 2011 Unidata Training Workshop, led by Unidata developer Michael James, were given the opportunity to test-drive the still-in-development CAVE (Common AWIPS Visualization Environment). For the workshop, the Unidata Program Center set up a single-server EDEX installation providing data for twenty CAVE workstations. Although the AWIPS II hardware configuration planned for operational forecast offices consists of multiple server machines supporting the CAVE workstations, this single-server/multiple-client AWIPS2 configuration shows promise for University computer lab settings.

Finally, the Unidata Program Center received its first unrestricted copy of the AWIPS II source code in late July. Program Center staff are in the process of evaluating the code and determining what steps will be necessary to create a package suitable for use by community members.

According to Scott Jacobs, acting chief of the NCEP Systems Integration Branch, NCEP Central Operations (NCO) continued its integration of NAWIPS functionality into AWIPS II during August."The main forucs is now to make the system stable and ready for forecaster testing in November," says Jacobs. "This includes collecting, decoding and storing more data types. We have emphasized getting more numerical model grid data ingested into the database."

Improvements to coordination between NCO and Raytheon (the primary AWIPS II software developer) have also been implemented, Jacobs continues. "We established a process to get automatic software updates from Raytheon via the NWS Office of Science and Technology. This allowed us to update quickly to version 11.7, the latest version at the time. Prior to this update method, it would take us over a month to integrate all changes from Raytheon with the SIB enhancements. Now, we can do continuous integration which allows us to submit our code to Raytheon in a more timely manner."

For additional information about the GEMPAK to AWIPS II migration, please see Unidata's GEMPAK/NAWIPS Migration Information.

According to Scott Jacobs, acting chief of the NCEP Systems Integration Branch, NCEP Central Operations (NCO) continued its work on the National Centers Perspective (NCP) client software in September in preparation for forecaster testing, currently scheduled for November. "The development of the required functionality is coming together and we will be conducting full system tests in October to suss out any final issues or bugs that would affect the user experience during their testing," said Jacobs.

Near the end of September, Raytheon and the NWS Office of Science and Technology (OST) delivered a 64-bit version of the client to the NCEP sites. Previous versions had run only in 32-bit mode."This version is for the users to test AWIPS I functionality," said Jacobs. "The testing has just begun, and a few configuration issues have been reported and fixed." Full stress testing of the 64-bit version is under way and the results will be reported to OST and Raytheon.

For additional information about the GEMPAK to AWIPS II migration, please see Unidata's GEMPAK/NAWIPS Migration Information.

Scott Jacobs, acting chief of the NCEP Systems Integration Branch, NCEP Central Operations (NCO), made a remote presentation on the status of AWIPS II development during Unidata's joint Users and Policy committee meeting in October. Jacobs' presentation highlighted the following details of the migration status:

• Full national deployment of AWIPS II to Weather Service offices and NCEP has been delayed by development issues.
• In the recent past, the Unidata Program Center has begun to receive snapshots of the AWIPS II development code base; this access will continue.
• A 64-bit version of CAVE (Common AWIPS Visualization Environment) has been developed, and will be included in the full national deployment.
• A 64-bit version of EDEX (the Environmental Data EXchange system, which supplies data to CAVE) is under development, but current plans call for its release after full national deployment of AWIPS II.
• Forecaster Integration Testing (FIT), which has been suspended while development issues were dealt with, will be resumed in November 2011.

AWIPS II demo at the UPC
(click for more)

Additional details are available in Jacobs'presentation slides.

After Jacobs' presentation to the joint committee, Unidata Program Center developer Michael James demonstrated the current state of the AWIPS II package for committee members. James, along with UPC staffer Jeff Weber, will be participating in AWIPS II training and field testing in the coming months. Although Unidata cannot release AWIPS II to community members until national deployment has taken place, work can begin on a custom Unidata AWIPS II installation package and on training materials tailored to the Unidata community.

For additional information about the GEMPAK to AWIPS II migration, please see the AWIPS II software page.

If you're interested in the AWIPS II project and attending the American Meteorological Society meeting in New Orleans, make some time in your schedule to stop by Unidata's booth in the exhibit hall. Unidata developer Michael James will be providing demonstrations throughout the day, and we'll also be hosting additional demonstrations by project leaders from the National Centers for Environmental Prediction.

Demonstrations by Unidata staff will be ongoing. NCEP staff are tentatively planning to join us in the Unidata booth at the following times:

• Monday: 6:00pm and 7:00pm
• Tuesday: 10:30am, 12:30pm, and 5:00pm
• Wednesday: 12:30pm, 3:00pm, and 5:00pm

Visit us in Booth 322 in the exhibit hall (map) for a peek at the AWIPS II development and to hear about everything else going on at Unidata.

Unidata is currently evaluating AWIPS II release 13.1.2, which includes a new unified grib decoder to supply both D2D and the National Centers Perspective with gridded data. With this decoder upgrade and the UPC's improved decoding of high-volume NEXRAD3 and CONDUIT feeds, the last major addition to AWIPS II before release is GEMPAK functionality. GEMPAK 7 will be released as an add-on to AWIPS II for the National Centers and the Unidata community.

With this in mind, Unidata is preparing the AWIPS II system for release to the public by the end of 2013, with an official release date to be determined. The software will be released in two parts:

1. Pre-built binaries for EDEX server and CAVE client installation.
2. AWIPS II Development Environment, including source code.

The source code will be maintained in a public github repository with an online documentation hub hosted at the Unidata website.

Visit the Unidata AWIPS II page for more information.

By Dr. Kevin H. Goebbert
Department of Geography and Meteorology
Valparaiso University

Students at Valparaiso University
(Click to enlarge)

The Valparaiso University Meteorology program strives to keep current with meteorological technologies so our students can maximize their educational opportunities. For the past seven years, the National Weather Service has been designing and building a new version of the Advanced Weather Interactive Processing System— used in weather forcast offices and national centers — called AWIPS II. This new software program will be made available to universities as a result of cooperation between Unidata, the National Centers for Environmental Prediction (NCEP), and Raytheon, the company developing the application. Recently, Unidata has been working on developing a methodology for deploying this versatile tool to campuses across the country, and has encouraged universities to apply for grants to aid in the development process. Valparaiso University's Meteorology program was awarded a 2012 Community Equipment Award grant to allow it to serve as a test school for the deployment of AWIPS II. Because our program has limited resources and infrastructure support, we provide an ideal “small school” test case for the deployment of AWIPS II. Specifically, the Unidata Equipment grant allowed our program to purchase the main server that will ingest data for AWIPS II. The server is a HP ProLiant DL360p Gen8 Server, with plenty of storage for all of the data.

Students in the Weather Center

The meteorology program at Valparaiso University is a long-time user of many software packages developed and maintained by Unidata. Our twenty-station weather center, which is regularly used for coursework and research, was recently updated to a dual-monitor Linux lab that supports many different software programs, including existing packages from Unidata and (eventually) AWIPS II. By allowing students to view meteorological data through a program widely used by government meteorologists across the country, the new server purchased with Unidata Equipment award funds will help the program to better prepare students for work in any meteorological job, especially those seeking National Weather Service employment.

Currently, we use a combination of Unidata software programs (including GEMPAK and the IDV) and Internet sources, some of which also also make use of Unidata software packages. Once full implementation is achieved, the Valparaiso University Weather Center will be upgraded to include the Common AWIPS Visualization Environment (CAVE), which will be heavily utilized in junior and senior coursework as students gain experience interpreting weather maps and making forecasts for the local region. AWIPS II and the CAVE viewer will present a one-stop shop for all weather-related data available through Unidata's Internet Data Distribution system (IDD), which can then be supplemented by other outside sources.

Additionally, we have been able to test the AWIPS II installation procedures and provide feedback to Unidata. As of this writing, we have not yet been able to successfully install and use the development versions of the software package, but we are learning a lot about the system through the testing phase. As a result of our participation in the testing process, we hope to be able to share our expertise to other universities as they do their own installations of the final AWIPS II package.

For additional information on the Unidata Community Equipment Awards, visit the Equipment Awards page.

In April, 2013, the first UPC-release of AWIPS II (version 13.2.1) was provided for a limited number of universities.  Now, Unidata welcomes additional sites to join this program. To apply, please write to support-awips@unidata.ucar.edu and provide the following information:

• Name of institution
• Who will be the point-of-contact for AWIPS II at your institution
• How many machines and/or users currently use GEMPAK/NAWIPS at your institution
• Are you able to host an EDEX data server for remote access (as opposed to only running AWIPS II internally at your institution)?

Please note that by joining this beta testing group you are able and willing to install and maintain the software and are willing to provide regular feedback to the Unidata Program Center about AWIPS II.

Unidata AWIPS II 14.2.1 is now available to download for beta testing.  AWIPS II runs on 64-bit Red Hat and CentOS 6, and is provided as a tarball archive and via remote yum repository.

See the Unidata AWIPS II software page for more information:

http://www.unidata.ucar.edu/software/awips2/

Full installation instructions can be found here:

http://www.unidata.ucar.edu/software/awips2/doc/install.html 

Unidata's AMS booth
(Click for more.)

This year's annual American Meteorological Society meeting was held January 4-8 in Phoenix, Arizona. We were happy to see many of the Unidata community members participating in the meeting at our booth in the exhibit hall, and to meet so many prospective community members at the AMS Student Conference.

With so much going on at the conference, we can't cover everything here. Instead, we present some highlights as recalled by UPC staff members who attended.

14th Student Conference

AMS Student Conference

The 14th AMS Student Conference had 620 attendees. As in recent years, Unidata had a table set up for the Student Conference Career Fair, held Saturday and Sunday evenings before the main conference opened on Monday. Unidata's table attracted many visitors, with students interested in data and software available from Unidata as well as Unidata'sSummer Internship program. (If you're interested, there is still time to apply.)

Continued AWIPS II Interest

AWIPS II in the Unidata booth

AWIPS II, the next-generation weather visualization and forecasting software developed by the National Weather Service, continues to gain traction in academic, research, and commercial organizations. Unidata works with the NWS and the National Centers For Environmental Prediction to bring a version of AWIPS II to universities, which don't have the same resources or requirements as the operational Weather Forecast Offices (WFOs) for which the system was designed.Unidata's version, which is currently being beta tested by several universities, offers research and education organizations the ability to run the system without requiring them to install the complex hardware configuration used in the WFOs' operational setting.

IDV Demonstrations

IDV Demonstrations

As always, Unidata's Integrated Data Viewer (IDV) was a big draw at the AMS conference booth. And while there is always someone on hand from the UPC to show off the IDV's capabilities, it's always nice to have community members stop by with some of their own data to show. This year,Brendan Hoch of Plymouth State University in New Hampshire (and former member of the Unidata Users Committee) showed off some of the graphics his program generates using the IDV and publishes on the university web site. If you're generating cool imagery with the IDV, bring it by the Unidata booth next year and let us show it on the big screen!

Python Symposium

The Symposium on Advances in Modeling and Analysis Using Python, now in its fifth year at the AMS annual meeting, was once again very well attended. UPC staff continue to explore ways to make Unidata technologies easier to integrate into scientific workflows built around Python, and were glad to participate in this year's symposium as attendees and speakers.

Other Unidata Presentations

Unidata Program Center staff also participated in several papers and posters presented at the conference:

• Mohan Ramamurthy co-chaired a session on Open Data Standards and Sharing
• Ward Fisher gave a poster on Accessing NetCDF4 Data in Python
• Jeff Weber had a poster on A New Approach to Sharing Curriculum and Data
• Ryan May and Sean Arms spoke on Exploring Client-Server Data Visualization using IPython and WebGL
• Yuan Ho and Julien Chastang spoke on Using Progressive Resolution to Visualize large Satellite Image dataset
• Jeff Weber gave a presentation on Typhoon Haiyan
• Michael James presented AWIPS II for the University Community: Unidata Program Center Update
• Sean Arms spoke at the New Ideas Forum: Networking and Idea Exchange for Student and Early Career Members
• Jeff Weber joined Embry Riddle University's Curtis James to speak on A Cloud-Based Mobile Weather Server to Support Emergency Response Meteorology Training and Operations
• Mohan Ramamurty co-chaired a session on Data Stewardship: Ensuring the Value of Data from Real-Time to Posterity

As always, the Unidata staff came away from the AMS annual meeting impressed by the camaraderie and fascinating research being done by Unidata community members and the geoscience community at large. We were also pleasantly surprised by the frequency with which we heard references to Unidata or our services while attending talks across AMS domains. If you've got a story to tell about how you're using Unidata software, data, or services that you think might help others advance their own research, drop us a note and let us help you spread the word.

If you are upgrading from 14.2.1 to 14.4.1, it is required that you completely uninstall your 14.2.1 instance before installing 14.4.1, to ensure that all required packages are installed (there were some updates to package names and release numbers that may cause conflict on yum groupupdate).

To update from 14.2.1 to 14.4.1 (as root):

edex stop
yum groupremove awips2-server -y
yum clean all
rm -rf /awips2/data/
yum groupupdate awips2-server -y
edex start

Manually removing /awips2/data/ is critical! This is required for the new version of Postgres to be installed cleanly!

Please reference the full installation instructions for your install or upgrade.

Some of the improvements and features in this release include:

• Server-side compression now reduces data transfer sizes by an order of magnitude.
• Client-side caching of maps and data resources (in ~/caveData/cache/) reduces querying and loading time.
• Postgres, Pypies, Qpid and EDEX start automatically on machine reboot.
• Python Data Access Framework allows users to write Python scripts to query remote EDEX maps and data.
• NCEP/Hydro and Upper Air menus now available in D2D
• D2D single-pane default perspective, toggleable to 5-pane "classic" view.
• NCP resources can now be loaded directly from the D2D perspective, making available the GEMPAK-like grid display without having to switch to the NCP perspective.
• Multi-Radar/Multi-Sensor (MRMS) full support for decoding and displaying gzipped grib2 files from NSSL.
• Support for decoding and displaying USPLN lightning data.
• NEXRAD3 menu includes full set of NEXRAD and TDWR sites, organized by site.
• NEXRAD3 mosaic capability renders a national composite of certain level 3 products queried from individual station files (functional but slow).
• Save/Load Bundle feature added for D2D, allowing users the option of saving bundles (as xml files) locally to disk or synced in ~/caveData to the EDEX server.
• McIDAS AREA files from the UNIWISC IDD feed decoded and displayed by the satellite plugins.
• NO CLIPPING OF GRIDS TO LOCALIZATION AREA
• Java 1.6 -> 1.7
• Fixed problem with how LDM writes CMC-REG grib1 messages which caused EDEX ingestGrib JVM to crash.
• Full south polar stereographic UNIWISC McIDAS area file support.
• Partial GVAR native projection support.
• D2D "Map Scale" menu now includes satellite-specific map projections (Arctic, Antarctica, Alaska-Hawaii-PR Regional, East/West CONUS, Global, etc.)
• D2D max number of frames increased to 400
• Ingest JVMs now run with maximum 4096M of memory.
• Qpid components updated from 0.30 -> 0.32
• Various fixes to EDEX-side hydro applications.

Editor's Note: Metropolitan State University of Denver (MSU) is one of a handful of universities of varying sizes that have incorporated AWIPS II into labs and classrooms in the past two years. MSU's experience shows that incorporating the new NWS technology into the educational mix is both possible and useful even for a small undergraduate program. This article outlines the MSU AWIPS II configuration and discusses some of the adjustments the program made to get the system running smoothly.

By Dr. Sam Ng
Chris Kimmett
Department of Earth and Atmospheric Sciences
Metropolitan State University of Denver

Professor Ng working with students in the Weather Analysis Techniques lab.

Thanks to a Unidata Community Equipment Award grant coupled with some local resources from our university, we (Professor Sam Ng and co-PI Chris Kimmett) at Metropolitan State University in Denver have been able to successfully bring the National Weather Service's AWIPS II forecasting system into the classroom. Professor Ng used AWIPS II alongside Unidata's Integrated Data Viewer as core tools in the Spring 2015 “Weather Analysis Techniques” course for undergraduate meteorology majors, and will continue using AWIPS II in the Fall semester and into the future.

Hardware Used

The AWIPS II hardware configuration specified by the NWS is designed for operational use in Weather Forecast Offices (WFOs). As such, it consists of several high-powered server machines making up the Environmental Data EXchange (EDEX), which ingests, processes, and serves real-time data to client computers running the Common AWIPS Visualization Environment (CAVE) software for display and analysis. Developers at the Unidata Program Center designed a slimmed-down EDEX server configuration consisting of a single server computer, which is more suitable for a non-operational setting such as a university classroom. MSU's EDEX server is configured as follows:

• HP ProLiant DL380p Gen 8
• 2 x 16 core Intel Xeon E5-2650 v2 @ 2.6 GHz
• 32 GB RAM ( 8 x 4 GB 1866 Mhz DDR3 SDRAM )
• 146 GB 15K HDD - Operating System
• 1.8 TB 10K HDD - Storage
• 300 GB SSD - Radar Data
• 1 Gigabit Network Interface
• RHEL 64 bit Server - version 6.6

This server, which cost roughly $11,000 to configure in early 2015, has proved capable ingesting all of the real-time data we desire (including NEXRAD3 Radar, GEFS, and 0.25° GFS model output) and supporting 20+ CAVE clients simultaneously in our weather lab.

In preparing our weather lab to take advantage of AWIPS II, we upgraded 27 existing workstations with extra memory and more capable video cards, giving us the following configuration:

AWIPS II CAVE and the IDV running on a lab machine.

• Dell Optiplex 960
• Intel Core 2 Duo CPU @ 3.0 GHz
• 8 GB RAM (800MHz DDR2 SDRAM)
• 160 GB 7.2K HDD
• 1 Gigabit Network Interface
• Nvidia 2GB PCIe x16 graphic card
• RHEL 64 bit Workstation - version 6.6

These lab workstations have proved capable of running both the AWIPS II CAVE client and Unidata's IDV simultaneously.

AWIPS II System Configuration

We began by installing the AWIPS II version 14.2.1 package created by Unidata. Installation itself was relatively straightforward, but getting AWIPS II running without crashing was a challenge. We found that several configuration files needed adjustment from the defaults to run smoothly in our environment.

First, we had to adjust the default pqact.conf file, which controls the actions of the Unidata Local Data Manager (LDM) software that is used by AWIPS II for data ingest. Ingesting model output for the Global Environmental Multiscale Model (GEM) from the Canadian Meteorological Centre (CMC) and the NAVy Global Environmental Model (NAVGEM) caused AWIPS II's GRIB file decoder to crash, due to errors in the GRIB tables. Removing these models (which we were not using in the classroom anyway) from the LDM configuration was as simple as commenting out the corresponding lines in our pqact.conf file, and solved the crashing problem.

Second, we adjusted the HEAP size request in the request.sh and wrapper.conf files to fix an issue we experienced when opening multiple CAVE clients. We found that increasing the MAX_MEM parameter in /awips2/edex/etc/request.sh to 4096 resolved a heap memory issue that would crash the ingestGrib process. We also found that changing the wrapper.java.maxmemory parameter in /awips/qpid/conf/wrapper.conf to 4096 resolved a heap memory issue the would crash the qpid process.

After making these adjustments, we have found the AWIPS II 14.2.1 system to be quite stable, with CAVE clients running for weeks at a time without rebooting or crashing. (Most recently, we haven't had a crash since the beginning of the Fall term in mid-August 2015.) We are indebted to Unidata Program Center staffers Michael James and Jeff Weber for helping us diagnose these issues and devise the appropriate solutions.

Note that our initial experiments with the AWIPS II 14.4.1 release in the Fall 2015 have not been as successful, because version 14.4.1 of the CAVE client software does not work with EDEX servers running version 14.2.1.

System Performance

Once the adjustments to our EDEX configuration were in place, the system has had no trouble ingesting the data we are requesting. Our CAVE client machines have similarly been up to the task of allowing up to 20 students at a time to access the system. We have found AWIPS II to be well suited for nowcasting and forecasting exercises in our Weather Analysis Techniques and Mesoscale Meteorology courses.

An Alternative Workflow

AWIPS II CAVE displayed on a Macintosh

The AWIPS II software environment runs only on Linux servers and workstations. While this is not an issue for the dedicated EDEX server or the lab workstations, this limitation was a bit of a stumbling block for Professor Ng, who uses a Macbook Pro to run classes and in daily use. As it turns out, it is possible to install the CAVE client software on a Macbook by running a Linux virtual machine -- in this case using the VMware Fusion software to create a virtual RedHat Linux workstation. Given sufficient network bandwidth, the CAVE client performs with no noticeable delays. In MSU's case, the network is fast enough even over a WiFi connection when using the campus Virtual Private Network (VPN). Your mileage may vary...

Unidata EDEX and CAVE version 15.1.2 are now available.

To update you can run the installEDEX.sh or installCAVE.sh scripts on your workstations (see full install instructions).

You can also manually update with:

yum clean all && yum groupupdate awips2-server

(or groupupdate awips2-cave for CAVE)

Our cloud-based data server (edex-cloud.unidata.ucar.edu) has been updated to the latest build and is serving a larger set of data than before, including a subset of MRMS grids.

Noteworthy EDEX updates:

• No more Qpid bottleneck preventing restarts
• MRMS NOAAport ingest support (enabled by default in pqact.conf)
• MRMS NSSL support using gunzip/dcgunzip (optional)
• More complete LDM pqact entries for various HDS grids
• Single file sourced from /etc/profile.d/ rather than multiples (true for CAVE as well)
• ECMWF global grid compositing

CAVE:

• New Models/VolumeBrowser menu and default bundles, including meso analysis (RTMA/URMA) and ocean models (ESTOFS/ETSS)
• Toolbar buttons to open NCP data manager, NSHARP, and Product Browser (NC data will open in its own D2D map editor)
• Satellite Menu UNIWISC imagery bundles
• GFE vcmodule muted, which now allows the GFE perspective to load correctly
• Re-enabled user-created colormaps
• Various menu organization changes

Editor's Note: This is part of a series of posts written by Unidata communications intern Larissa Gordon, highlighting new activities and interesting projects undertaken by software developers at the Unidata Program Center.

Plotting a Grid using matplotlib, numpy, and basemap
(Click to enlarge)

By Larissa Gordon

Unidata is pleased to announce the availability of the Python Data Access Framework (DAF). The DAF provides access to an AWIPS II Environmental Data EXchange (EDEX) server directly from Python code. Created by Unidata Program Center software engineer Michael James, the DAF strengthens Unidata's AWIPS II offerings by making it easier to retrieve data from the AWIPS II data storage system from outside the Common AWIPS Visualization Environment (CAVE).

Users have always been able to request real time NCEP/NWS weather data using AWIPS II, but now, with the addition of the Python DAF, users can request this data using only simple python commands.

The python DAF is very versatile because it is a client Python Module that requests data from a remote EDEX server. It does not depend on the CAVE client, or on any other AWIPS II technologies. Therefore, the Python DAF can be installed without the other packages that are a part of AWIPS II. This allows it to be used on any computer, even if that computer cannot run the AWIPS II software.

The Python DAF gives users the option of requesting data from the THREDDS catalog or the AWIPS II catalog. Similar to how Siphon requests data from THREDDS, the Python DAF allows users to query specific times, levels, and parameters from AWIPS II.

Plotting Surface Observations with MetPy

Users can easily render the retrieved data objects with any number of Python packages; MetPy, Matplotlib, and Cartopy to name a few. In addition, the Python DAF supports various data sets including gridded models, upper air profiles, METAR/Synoptic obs, NEXRAD Level 3 radar, and various text-based observations (ACARS, Profiler, Marine obs, AIREP/PIREP).

To introduce users to the new Python DAF, James has created and published a set of Jupyter/iPython notebooks on Github.com. Using the notebooks users are able to learn how to request and then render the various AWIPS II data sets.

The Python DAF example notebooks are available here.

Teaching Python with Jupyter notebooks is is well suited to the classroom because the notebooks are interactive scripts. Even better, using the new Python DAF, users no longer need to download and locally-store meteorological data sets. Users can simply request the real-time data sets they would like to explore from the remote EDEX data server whether it be in a classroom context or for research.

As Python becomes increasingly popular among the geoscience computing community, it is important for software engineers to tailor the software packages already in place to make Python more applicable. Michael James has allowed for this by making NCEP/ NWS weather data more available thanks to the new Python Data Access Framework.

NWS meteorologists demonstrating AWIPS II
(Click to enlarge)

By Dr. Clark Evans
Atmospheric Science Program
Department of Mathematical Sciences
University of Wisconsin, Milwaukee

Thanks to a 2015 Unidata Community Equipment Award grant, the University of Wisconsin-Milwaukee Atmospheric Science Program has been able to acquire a dedicated AWIPS II Environmental Data EXchange (EDEX) server and three Common AWIPS Visualization Environment (CAVE) workstations. The machines were deployed in our Atmospheric Science computer lab in mid-August 2015, just prior to the start of the 2015-16 academic year. Over the past year, our program has seen maximum benefit from these resources in two areas: strengthening our partnership with the Milwaukee/Sullivan, Wisconsin National Weather Service Weather Forecast Office (MKX) and as a tool to enhance student development outside of a formal classroom setting.

Historically, our program has had a strong working relationship with the Milwaukee/Sullivan Weather Forecast Office that has included research collaborations, student internship placement, and the StormReady program. The deployment of AWIPS II EDEX and CAVE systems in our lab has only strengthened that relationship. In mid-October 2015, MKX Warning Coordination Meteorologist Tim Halbach and Meteorologist Intern Ben Herzog visited our program and provided an approximately two hour AWIPS II CAVE demonstration to a group of students, instructors, and faculty. Since then, we have been working with Jerry Wiedenfeld from MKX to localize our AWIPS II deployment to WFO MKX.

Perhaps more important, however, is the enhanced facilitation of student development fostered by the new equipment. This is particularly beneficial to a program such as ours, where approximately 20 percent of our undergraduates and 35 percent of our graduate students gain employment with the National Weather Service after graduation. Of particular note, during the Spring 2016 semester, a senior undergraduate student interested in National Weather Service employment, Alec Muniz, completed a Capstone experience related to using AWIPS II under my supervision. Beyond an increased familiarity with and ability to use AWIPS II CAVE, Alec prepared a comprehensive “Introduction to AWIPS II” training module, covering D2D, Nsharp, GFE, and WarnGen. He used this module to train one of our senior lecturers, Bart Adrian, in the use of AWIPS II CAVE, furthering the culture of servant leadership that we attempt to have permeate our program.

Undergrad Alec Muniz with senior lecturer Bart Adrian

“When I heard last year that the Atmospheric Science Program was going to be receiving AWIPS II, I was very excited for the opportunity. I used AWIPS II at the WFO Sullivan/Milwaukee while I was a student volunteer, and being able to continue gaining experience was something I thought not possible outside of the NWS,” say Muniz. “Using AWIPS II as a beginner can be overwhelming and intimidating — I know it was for me — and I wanted to create a manual that would be an easy starting point for others. My goal was for someone with no experience with AWIPS II to be able to sit down and have something that they could easily follow and reference while they explored AWIPS II. This document is shared with the students in the program and I hope this can assist them learn the software and, hopefully, to encourage them to add what they learn to the document in order to help others.”

The training manual itself resides in the cloud, where all students and faculty have access to learn from, add to, and revise the information therein. Believing that the entire Unidata community may benefit from this effort, we are pleased to share this module with the community. The most current version of the UWM AWIPS II Introduction and User's Guide is availble on the University of Wisconsin-Milwaukee's cloud storage service.

Says senior lecturer Bart Adrian, “I enjoyed being taught some of the ropes on the AWIPS II system by my former student Alec Muniz. Having access to the system presents a great opportunity for our students to become acquainted with the state-of-the-art tools of the National Weather Service. I'm looking forward to introducing my future students to AWIPS II in the classroom next year!”

Over the course of the next year, we intend to deploy the Mac OS and Windows AWIPS II CAVE binaries on the machines within our Atmospheric Science computer lab and at our Innovative Weather program's headquarters, respectively. The former will enable for a larger number of students at a time to make use of AWIPS II, which otherwise would serve as an impediment to our program's utilization of AWIPS II within course offerings such as Atm Sci 690, Daily Weather Discussion, and Atm Sci 360/361, our Synoptic Meteorology I/II sequence. The latter will allow for students to gain experience using AWIPS II CAVE in a real-world forecasting operation much like that which they would experience as operational forecasters with the National Weather Service.

Unidata Community Equipment Award grants make funds available to colleges and universities to puchase equipment or cloud-based computing services that will enhance their participation in the Unidata program. For additional information on program, visit the Equipment Awards page.

DHR composite, active warnings, and surface winds.

CAVE 16.1.5 requires a full uninstall of previous versions (14.4 or 15.1), which is handled fully by the shell script installCAVE.sh. No client files will be overwritten since user bundles, colormaps, and other configuration files are located in ~/caveData/.

• To install CAVE for Linux, download and run installCAVE.sh

• To install EDEX for Linux, download and run installEDEX.sh

What's New in 16.1.5

• Switching the localization site in CAVE Preferences now updates the side-view maps to the new WFO area of responsibility.
• New general "awips2-localization" RPM obsoletes site-specific "awips2-localization-OAX" with support for all WFOs.
• Simply running "cave" will launch the program, running the full path name /awips2/cave/cave.sh is no longer required.
• Full support for UNIWISC McIDAS AREA files, including new support for
  • 20km Global Rectilinear IR and WV
  • 30km Global Mollweide IR and WV
• Support for HIMAWARI-8 McIDAS AREA files
• Fix for GINI projection error in decoding of Supernational composites
• New colormap defaults for IR and WV imagery; GEMPAK colormaps also have been added
• New default map projections include Global Mercator, Global Mollweide, and Regional displays for Alaska, Africa, Antarctica, Aus/NZ, Europe, Hawaii, Japan, Pacific Ocean, Puerto Rico, and South America.
• Various bug fixes for the GFE Perspective
• New package "awips2-python-gfe" makes available various Python functions used by GFE which were previously absent.
• Hold-right-click menu on product stack now includes Interpolation toggle.
• Default display of FNEXRAD composites using data interpolation and decreased tiling for clearer data display.
• Two-pane toggle added to D2D right-click menu (along with existing 4-pane toggle)
• Two-pane displays are now oriented side-by-side rather than above-and-below.
• Reconfigured D2D NSHARP plugin to display selectable stations (NCP-style).
• Reconfigured image export with default timestamp in file name to prevent overwriting. Files are now exported to /awips2/export/<USER>/
• GFS20 replacing GFS40 in the Models menu
• New METAR precip total plugin colorizes rainfall amounts
• Metar surface winds bundle added to Surface menu
• Reorganization of Surface WATCH/WARN bundles, loadable as TStorm/Tornado, Flood, Marine, Extreme Wines, and "all". Bundle polygon contrast has been increased as well.
• Redbook graphics bundles displayed with higher contour contrast
• Complete regional imagery Satellite sub-menu with full product set for Alaska, Hawaii, Puerto Rico, and Arctic composites.
• New Satellite sub-menus for NOAAport GINI products (GOES East, GOES West, GOES E/W 10 km Composite, 24 km National Composite, 24 km NH Composite, Sounder Imagery)
• NPP VIIRS Near Constant Contrast bundle composites all available areas for product display.
• ESRL/GSD experimental 3 km HRRR now supported (HRRRX), ingesting on edex-cloud.unidata.ucar.edu (though still missing some parameters)
• Removed reporting of internal connections in httpd logs (responsible for ~90% of file size and unimportant). Enabled logging of DNS-resolved hostname, if available, for CAVE and python-awips. It's now easier for EDEX administrators to monitor who is connected to their server and how much data is being transferred by looking at log files in /awips2/httpd-pypies/var/log/httpd/.
• Default purge rule for satellite products preserves 96 frames (12 day archive) for Global IR/WV composites, and 288 frames (24 hours) for FNEXRAD images.
• GFE server support available for all WFOs, not just OAX (requires a change to setup.env and edex_camel restart - this does not mean support for all server sites simultaneously).
• Binlightning, nctaf, and nctext decoders re-enabled.
• Removed all metar/synop-to-Shef processing due to performance problems.
• CAVE operates strictly in practice mode, preventing WarnGen products from being pushed to the connected EDEX server.

By Dr. Josh Durkee
Meteorology Program
Department of Geography and Geology
Western Kentucky University

Western Kentucky Student using AWIPS II
(Click to enlarge)

The Bachelor of Science Meteorology Program at Western Kentucky University, housed within the Department of Geography and Geology, is currently the only meteorology program in Kentucky and Tennessee that meets all Federal Civil Service requirements for employment by the National Weather Service and enables broadcast meteorologists to immediately pursue the “Certified Broadcast Meteorologist” program of the American Meteorological Society upon graduation. The program is facilitated by a variety of atmospheric science centers and laboratories, including the College Heights Atmospheric Observatory for Students (CHAOS), the Kentucky Mesonet, and the Kentucky State Climate Center. Faculty and students within the WKU Meteorology Program also maintain close working relations with the National Weather Service-Louisville and other nearby offices for lectures, workshops, storm surveys, and student employment.

The emphasis of the WKU Meteorology Program is on advanced theoretical and applied coursework, coupled with student engaged, hands-on active learning with meteorological instruments, field work, and data visualization. Students undergo extensive training in each course using a variety of Unidata and other software for data visualization regarding forecasting and research applications.

With the goal of continuing to professionalize the learning experience at WKU, my colleague Dr. Eric Rappin and I sought a Unidata Community Equipment Award for support in implementing AWIPS II for a contemporary/enhanced approach toward meteorological education and research. During the fall 2015 semester, the WKU Meteorology Program installed a new EDEX server and three CAVE clients to test the AWIPS II system. Upon installation, a variety of students, faculty, and staff explored the software capabilities to provide feedback to and seek further support from Unidata for potential improvement of subsequent release versions.

NWS forecaster Mark Jarvis leading AWIPS II workshop

In March 2016, Mark Jarvis, Lead Forecaster with the NWS in Louisville visited the WKU Meteorology Program to provide an AWIPS II training workshop. Participants were afforded the opportunity to look beyond the face value of AWIPS II and learn the deeper capabilities and powerful forecast visualization options of the software.

“The National Weather Service AWIPS II Workshop was an excellent preparatory experience for use of the software,” said Pierce Larkin, a 3rd year B.S. Meteorology Major who attended the training. “The information was detailed, informative, and critical to understanding the software when we gained access to it at WKU. The seminar gave me insight into how to effectively use the software, and how to troubleshoot issues that I have. Moreover, as a volunteer at the NWS-Louisville office, having AWIPS II experience has given me a great running start at the office!”

Another exciting exercise related to AWIPS II implementation at WKU included examination of data visualization design and output with a Tobii X2-60 Eye Tracker. In a pilot study, we worked with experienced and non-experienced AWIPS II users to evaluate the software design and usability, as well as the package's visualization of fundamental meteorological output. This eye-tracking technology provided the ability to quantitatively measure an individual's points of interest and precise eye movements with 2D or 3D visualizations. Specific eye movements were statistically correlated to the attention path demonstrated by student observers. In a forthcoming formal study, we plan to provide feedback from using eye-tracking approaches toward understanding software design and meteorological education and understanding.

Visualization of eye movements when using AWIPS II

“The eye tracker experience revealed to me that I was understanding the meteorology, weather analysis, and data visualization in a way that was becoming natural. This was made possible with the well-designed graphic quality of AWIPS II, which enabled map analysis to be a relatively easy and simple process,” said 3rd year B.S. Meteorology Major Christopher Reece, who participated in the pilot study.

Looking ahead, our goal is to implement additional methods of data retrieval, including the use of the onsite NOAAPort system at WKU and Python data access methods. Currently, students and faculty are heavy users of the Unidata software suite (e.g., IDV, GEMPAK, McIDAS) for daily forecast discussions, instruction, class assignments and exams, and research. The overarching goal is to compliment these utilities by providing access to AWIPS II as a fundamental weather and climate analysis tool at the start of the 2016-17 academic year.

Unidata Community Equipment Award grants make funds available to colleges and universities to puchase equipment or cloud-based computing services that will enhance their participation in the Unidata program. For additional information on program, visit the Equipment Awards page.

By Brendon Hoch & Jason Cordeira
Department of Atmospheric Science & Chemistry
Plymouth State University

Students a Plymouth State University (click to enlarge)

Thanks to a 2015 Unidata Community Equipment Award grant, combined with matching funds from Plymouth State University, students working in the weather lab in the Department of Atmospheric Science & Chemistry now have 21 new workstations on which to use the Integrated Data Viewer (IDV) and AWIPS II.

During the 2015-16 academic year, the workstations were used heavily by students in a variety of classes, including Synoptic Meteorology, Atmospheric Physics, Instrumentation & Observations, Mesoscale Meteorology, Numerical Weather Prediction, Satellite & Radar Meteorology, and our meteorological computer application class, which provides in depth instruction with IDV and ISL. Furthermore, students used the workstations to produce scholarly research in areas such as atmospheric rivers, orographic precipitation, and explosive extratropical cyclone intensification over the North Pacific.

Students also used the machines to produce real time ski and showboarding forecasts, which benefitted Plymouth State athletic programs. Chris Hohman is a rising sophomore who worked on providing forecasts for skiing and snowboarding enthusiasts as part of a collaboration with a private company called OpenSnow. “The environment to work in was unique; it's specifically designed for meteorology work. Instead of being a regular computer cluster, the monitors and additional computing power made it easy to interpret multiple applications at once, including web-based applications and IDV,” Hohman says of the lab setup. “I never felt frustrated or slowed down by the tools in the computer lab, they are without a doubt highly capable of running multiple programs that I needed to create quality forecasts.”

Rob Megnia, a graduate student who also worked on providing forecasts for skiing and snowboarding enthusiasts, notes, “[The] dual monitor PCs give students a comfortable and resourceful work space. The dual screen PCs proved to be very convenient for blog forecasting ski and snow conditions for OpenSnow. I was able to write on one screen while analyzing forecast products on the other.”

Additionally, students with financial aid awards from the U.S. Department of Education's Federal Work Study Program had the opportunity to use the IDV on the lab workstations to develop new graphics products for the Plymouth State Weather Center (http://vortex.plymouth.edu). This has been a useful project as we continue to phase out maps produced by legacy WXP software.

The Unidata grant and matching funds purchased new Dell Optiplex 9020 workstations with NVidia GeForce GTX 745 4 GB video cards, allowing us to meet the video requirements for AWIPS II and to allow IDV to function more efficiently in a dual monitor environment. The video card features one each of HDMI, VGA, and DVI port, making it easy to integrate with the existing dual monitor configuration at each workstation. Each system has a i5-4590 Quad Core 6 MB 3.3 Ghz processor and 8 GB of 1600 MHz DDR3 memory and a 500 GB hard disk.

Originally, it was envisioned that each PC would need to be able to run both Microsoft Windows (to support the majority of university installed software) and CentOS Linux (to support AWIPS II). Two machines were configured with an additional solid state drive to test the image deployment process and dual boot functionality. During system configuration testing, Unidata announced that it would be providing an AWIPS II client for Microsoft Windows. Once the client became available, it was tested for compatibility with both Unidata's cloud EDEX server and our own local EDEX server. Initial results indicate that students are able to use the Windows client for CAVE to perform a variety of meteorological analyses. By using the Microsoft Windows CAVE client, the amount of effort to manage and deploy software will be greatly reduced by eliminating the need to keep two operating systems current with security patches and other updates. Faculty anticipate integrating AWIPS II into the curriculum shortly.

The previous generation of lab computers were transitioned for use by our graduate students for email/word processing or relocated to other classrooms for student needs that require minimal computing power. With this repurposing of the older machines, the overall life cycle for each computer will be seven to nine years, maximizing our technology investment.

The access to dedicated computer resources for student forecasting activities in the meteorology lab has been extremely useful. The number of high-power computers ensures that there is always one available for student work, and the resources afforded by the machines made the meteorology lab a desirable place for students. The computers gave students access to meteorological applications and would not necessarily be available on their personal PCs.

In the near future, the computers will continue to support courses and student research. Our program just received a three-year grant from the National Science Foundation to study Cold-air Damming in Northern New England. Both graduate and undergraduate students will use the lab facility to access observational data and numerical model simulations on our server to support this research. In addition, Plymouth State University is moving toward a more interdisciplinary model of education that incorporates real-world problems into the classroom. The computer lab is already used across disciplines; an Environmental Science graduate level GIS course used the facility this year and we anticipate similar collaborations will continue in the future.

Unidata Community Equipment Award grants make funds available to colleges and universities to puchase equipment or cloud-based computing services that will enhance their participation in the Unidata program. For additional information on program, visit the Equipment Awards page.

Unidata AWIPS 16.2.2 is a new set RPMs which are designed to fully update existing 16.1.5 CAVE and EDEX installations. To update CAVE or EDEX, simply run the installCAVE.sh or installEDEX.sh scripts.

16.2.2 Release Notes

• CAVE is supported on CentOS/Red Hat 7 (x86_64), installed via the same installCAVE.sh script (the script will determine CentOS version and point to http://www.unidata.ucar.edu/software/awips2/doc/el7.repo for el7)
• EDEX is not available for CentOS/Red Hat 7 at this time.
• CAVE is now installed as two packages (awips2-cave, awips2-cave-wrapper) rather than as individual component packages.
• GOES-R product sub-menus added to D2D Satellite menu.
• National ingest of BufrUA soundings (previous versions filtered location to WFO localization).
• AlertViz has been replaced by AlertView.
• CAVE added to Gnome menus under Applications > Internet > AWIPS CAVE.
• LDM can be started and stopped by both awips and root users, solving the process ownership conflict which happened when root issued an ldmadmin start or stop after the service was already started by user awips.
• NAWIPS-like colorized surface temperature bundle added to the Surface Obs menu.
• EDEX purge scours raw data to within 1 hr, freeing more than 100 GB of space (LDM scour would scour to within 24hrs).
• Updated NUCAPS NSHARP plugin to color sounding locations based on quality control flag.
• Product ingest fix for nctext decoder.
• NEXRAD Display plugin for D2D with selectable stations to load level 3 site products and map bundles.
• NCEP's Day-Night Terminator overlay resource now available to plot in D2D perspective.
• Every WFO and 88D location now has its own map scale, WFOs selectable in the D2D Map Scales dropdown, 88Ds selectable via Radar > NEXRAD Display.
• Restart button added to CAVE Localization Preferences, made active when site or localization info is updated.

CAVE 16.2.2 status

• CentOS/RH 6 x86_64 - installCAVE.sh
• CentOS/RH 7 x86_64 - installCAVE.sh
• Mac OS X - awips2-cave-16.2.2.dmg
• Windows 64-bit - 16.2.2 STILL UNSUPPORTED, use existing 15.1 releases from http://www.unidata.ucar.edu/software/awips2/doc/install.html

Relevant links

• http://unidata.github.io/awips2/
• http://www.unidata.ucar.edu/software/awips2/doc/install.html
• http://python-awips.readthedocs.io/en/latest/
• http://www.unidata.ucar.edu/software/awips2/installCAVE.sh
• http://www.unidata.ucar.edu/software/awips2/installEDEX.sh

Notes

• You may need to remove ~/caveData (or ~/Library/CAVE on Mac) after updating CAVE (if you are re-connecting to the same EDEX server your bundles and other customizations will be re-synced).
• As root you may need to re-source /etc/profile.d/awips2.sh
• Support requests should be sent to support-awips@unidata.ucar.edu

At the student career fair
(Click for more.)

This year's annual American Meteorological Society meeting, was held January 22-26 in Seattle, Washington. We were happy to see many of the Unidata community members participating in the meeting at our booth in the exhibit hall, and to meet so many prospective community members at the AMS Student Conference.

With so much going on at the conference, we can't cover everything here. Instead, we present some highlights as recalled by UPC staff members who attended.

16th Student Conference

The 16th AMS Student Conference had more than 700 attendees. As in recent years, Unidata had a table set up for the Student Conference Career Fair, held Saturday and Sunday evenings and Monday before the main conference exhibition hall opened. Unidata's table attracted many visitors, with students interested in data and software available from Unidata as well as Unidata's Summer Internship program. (If you're interested, there is still time to apply.)

Standing-room only at the Python Symposium keynote

Python Everywhere

The Symposium on Advances in Modeling and Analysis Using Python, now in its seventh year at the AMS annual meeting, was once again very well attended. Unidata's Ryan May, who is the lead developer on the MetPy project, presented the symposium's keynote address to a standing-room only crowd. Says Ryan, “Python's popularity seems to have grown further; almost everyone I talked to (an admittedly biased sample) were using it or had plans to learn.” (You can find a recording of Ryan's keynote talk on the conference web site.) Other presentations by Ryan and UPC developer Julien Chastang were also quite popular.

One event of special interest was the AMS Short Course on Interacting with Radar Data in The Cloud. This workshop focused on getting attendees up an running using Jupyter notebooks interacting with NEXRAD data available in Amazon's cloud-based storage. According to Ryan (who somehow found time to help out at the workshop alongside his other presentation duties), “ Attendees seemed to enjoy getting hands on setting up an AWS instance and running their own jupyter notebook server, as well as playing with radar (and other) data using Python.”

AWIPS at the Unidata booth

AWIPS Update

You're probably aware that Unidata has been working with the National Weather Service (NWS) and the National Centers for Environmental Prediction (NCEP) to make the Weather Service's next-generation Advanced Weather Interactive Processing System (AWIPS) available to the university community. While the system was desiged for use at NWS Weather Forecast Offices and National Centers, Unidata has been working with the system's primary developers to create a configuration better-suited to use by university educators and researchers. Unidata's version of the client software can run on relatively modest computing equipment; Unidata has also made it possible for non-NWS users to gain testing access to an AWIPS Environmental Data Exchange (EDEX) server running remotely in the Microsoft Azure cloud, removing the need to configure the AWIPS data server before even trying out the client software. More than 60 universities have installed Unidata's version of AWIPS, and many of them are also using the cloud-based EDEX server.

Conversations with community

Catching Up With the Community

As always, AMS was a great time for Unidata staff to interact and catch up with community members. Unidata's booth on the exhibition floor was busy, and there were many productive conversations with friends and colleagues. Unidata staff came away from the AMS annual meeting impressed by the camaraderie and fascinating research being done by Unidata community members and the geoscience community at large.

If you missed the meeting, or just missed some of the many presentations given by Unidata folks this year, you can find the abstracts in the schedule in the Unidata Staff at AMS 2017 Meeting post on the News@Unidata blog.