Tricks for wgrib2 2/2013 old: compatible with wgrib new: wgrib2 only (1) How do I convert the entire file to binary? wgrib2 grib_file -bin out.bin (2) How do I get a verbose listing of the 20th record only? wgrib2 -V -d 20 grib_file (3) How do I extract the 20th and 30th records? new: wgrib2 grib_file -match "(^20:^30:)" -bin binary_file new: wgrib2 grib_file -for "20:30:10" -bin binary_file old: wgrib2 grib_file | egrep "(^20:|^30:)" | wgrib2 -i grib_file -bin binary_file (4) How do I extract the first 10 records as a grib file? new: wgrib2 grib_file -for 1:10 -grib new_grib_file old: wgrib2 grib_file | head -n 10 | wgrib2 -i grib_file -grib new_grib_file (4') How do I extract records 34-66 from a file as a grib file? new: wgrib2 grib_file -for 34:66 -grib new_grib_file old: Here is an example of an awk filter. wgrib2 -s grib_file | awk '{if ($1 > 33 && $1 < 67) print $0}' FS=':' | wgrib2 -i grib_file -grib new_grib_file FS=':' implies that arguments are separated by colons. As a result, the record number is the first argument ($1). The variable "$0" is the entire input line. Isn't awk fun? (4''') How do I extract records 10,12,19 from a grib file wgrib2 old_grib -match '^(1|12|19):' -grib new_grib The ^ is a special regular-expression character that matches the start of the line. The () denotes an expression and the vertical bar is the OR operator. See you book on regular expressions for details. (5) How do I remove the 2+i*8 th record from the file? You can do it with an AWK filter, but here is what I used. Remember -if is terminated by -fi or an output. wgrib2 in \ -if_n 1::8 -s -grib out\ -if_n 3::8 -s -grib out\ -if_n 4::8 -s -grib out\ -if_n 5::8 -s -grib out\ -if_n 6::8 -s -grib out\ -if_n 7::8 -s -grib out\ -if_n 8::8 -s -grib out (6) Some NCEP polar stereographic/lambert conformal files say that dx=0 and dy=0. Some NCEP files encode the grid spacing but set a flag specifying that the grid spacing is not given. In this case wgrib2 and wgrib display a grid spacing of zero. You can set the flag correctly by wgrib2 -set_flag_table_3.3 48 in.grb -grib out.grb (7) CMC files have some variables which don't have a name. wgrib2 doesn't include the local extentions from CMC. The CMC grib tables follows the NCEP grib table except where they don't. So you may be able to get by by having wgrib2 use the NCEP tables. This can be done changing the center flag to NCEP (7). wgrib2 -set center 7 (CMC grib file) obsolete: wgrib2 -set_center 7 (CMC grib file) Warning, "-set center 7" changes the center of the in-memory grib message. If you write out the grib message, it will now have the center set to 7! wgrib2 -set center 7 (CMC grib file) -match 'HGT' -grib hgt.grb Note that hgt.grb will be labelled as coming from NCEP (7). If you are using g2ctl, you can use the NCEP table by changing $wgrib='wgrib'; to $wgrib='wgrib -set center 7'; in the g2ctl source code. (8) I get *** ERROR: not handled by to_we_sn_scan *** The default mode of wgrib2 is to convert the grid to WE:SN order. The code to convert from the native order to WE:SN hasn't been tested or written. This happens with grids with a variable number of lat or lon points. In order to use this file, you have to use the file in raw-order mode. I.e., use "-order raw" on the command line. When you dump the data, the grid points will be output without any translation of the order. wgrib2 -order raw -d 1 grib_file -bin out.bin (9) Getting more information The option -v and -v2 should be tried. Some of the options give a more verbose output. (10) I am generating a text dump (CSV) and the size of the file is really large. You can set grid point values to "undefined" and then they won't print out. wgrib2 grib_file -undefine out-box 30:40 -10:10 -csv out.txt The grid points outside of the box are set to undefined. The -csv options tries to print the entire grid but finds only the values between 30N-40N and 10W-10E are defined. (10') I am generating a text dump (CSV) and the size of the file is very large and I need all the points including undefined. The -ijbox will print out all the values within a box, including the undefined valued. (11) I tried using the small_grib option but the files are bigger than the original. By default, The small_grib option writes a regional subset in grib1-packing. This is faster but produces larger files that the jpeg packing. To make smaller files, use the -set_grib_type_type jpeg before the -grib_out option. (12) I want to modify some existing grib files. For example, I want to make sure the RH is positive. Suppose my file only has 1 record. 1) Save the data in a text file, tmp.out. -sh-3.00$ wgrib2 test.grb -no_header -text tmp.out 1:0:d=2009030712:SPFH:2 m above ground:6 hour fcst: 2) Change the data file, tmp.out, using the program of your choice. 3) Use the import_text option to read in new data values and use the -grib_out to write the values to a grib file. Note that you can not use the -grib option. -sh-3.00$ wgrib2 test.grb -no_header -import_text tmp.out -grib_out new.grb 1:0:d=2009030712:SPFH:2 m above ground:6 hour fcst: (13) My software won't handle the level "30-0 mb above ground". How do I change the level to "15 mb above ground"? wgrib2 in.grb -if ":30-0 mb above ground" -set_lev "15 mb above ground" -fi -grib out.grb Requires 5/09 version of wgrib2. (14) I have a list of options that I frequently use. How do I reduce the typing? (14') The option list is getting long and hard to read. You can place the commands in a file. ------- my_list ------ -grid -nl -packing -nl -stats ----end of my_list---- -sh-3.00$ wgrib2 gfs.t18z.master.grbf00.10m.uv.grib2 `cat my_list` 1.1:0:grid_template=0: lat-lon grid:(720 x 361) units 1e-06 input WE:NS output WE:SN res 48 lat 90.000000 to -90.000000 by 0.500000 lon 0.000000 to 359.500000 by 0.500000 #points=259920: :packing=grid point data - jpeg2000 compression: :ndata=259920:undef=0:mean=-0.0242625:min=-24.13:max=26.44:cos_wt_mean=-0.228323 1.2:0:grid_template=0: lat-lon grid:(720 x 361) units 1e-06 input WE:NS output WE:SN res 48 lat 90.000000 to -90.000000 by 0.500000 lon 0.000000 to 359.500000 by 0.500000 #points=259920: :packing=grid point data - jpeg2000 compression: :ndata=259920:undef=0:mean=0.672931:min=-18.61:max=23.4:cos_wt_mean=0.788584 (15) I want to write the second records of a decoded grib-point data to stdout without the inventory. Use the -inv option to send the inventory to a read-only file and the symbolic file "-" which represents stdout. wgrib2 -d 2 input.grb2 -inv /dev/null -text - (16) Masking data: I want set all oceanic values to undefined. Suppose LAND is a grib file with the land mask (land=1, water=0) with the same grid as the data file (data.old) Make a new file with the land mask as the first record. cat LAND data.old > data.tmp wgrib2 data.tmp \ -if '^1:' \ -rpn '1:==:sto_1' \ -fi \ -not_if '^1:' \ -rpn 'rcl_1:mask' \ -grib_out data.new Line 2: if the first record then Line 3: see if land mask is equal to 1, save in register 1 Line 4: endif Line 5: if not the first record then Line 6: recall register 1, apply mask operator Line 7: write to data.new, endif (17) Make a daily max from the TMAX of the 00Z-06Z, 06Z-12Z, 12Z-18Z and 18Z-24Z wgrib2 data >inv grep ":d=$YYYYMMDD" data_22_55.txt wgrib2 IN.grb -lon 22 58 > data_22_58.txt wgrib2 IN.grb -lon 23 33 > data_23_33.txt In the above example, the file is being decoded 3 times. Time can be saved by decoding once. wgrib2 IN.grb -lon 22 56 -lon 22 58 -lon 23 33 >data.txt (28) I have an irregular grid (130). How do I get the lat-lon of the points? (a) The -grid option will print out the first the lat-lon values for the first 300 points. (b) wgrib2 irr_grid.grb2 -d 1 -no_header -rpn rcl_lat -text lat.dat \ -rpn rcl_lon -text lon.dat (c) wgrib irr_grid.grb2 -d 1 -rpn 0 -spread latlon.txt (29) How do I remove duplicate records? The following routine removes records that have the same inventory. wgrib2 IN.grb -submsg 1 | unique.pl | wgrib2 -i IN.grb -GRIB OUT.grb Note: (a) for messages with submessages, only the inventory of the 1st message is checked. i.e. if the 1st submessage matches and the second submessage doesn't, the entire message is removed. (b) order is preserved (c) submessage structure is retained, see (a) (d) note that on the command line, "GRIB" is capitalized ----------------------- unique.pl ------------------------ #!/usr/bin/perl -w # print only lines where fields 3..N are different # while () { chomp; $line = $_; $_ =~ s/^[0-9.]*:[0-9]*://; if (! defined $inv{$_}) { $inv{$_} = 1; print "$line\n"; } } --------------------- end unique.pl ---------------------- (30) I want to use the editor to change the 2 m RH fields in a file Here is a simple way a extract and change a single grib message. wgrib2 IN.grb -match ":RH:2 m above ground:" -text rh.txt -grib rh.grb .. extract 2m RH in text and as a grib file edit rh.txt .. modify the grid point values wgrib2 rh.grb -import_text rh.txt -set_grib_type c3 -grib_out new_rh.grb .. write a new grib file with modified values Suppose you want to keep the original file and change a specific record wgrib2 IN.grb -match ":RH:2 m above ground:" -text rh.txt edit rh.txt wgrib2 IN.grb -if ":RH:2 m above ground:" -import_text rh.txt -fi -set_grib_type c3 -grib_out OUT.grb The above sequence will lose the submessages and waste time re-encoding the grib message. It is possible to save not reencoding. (31) I wanted to find the min TMP2M for a given year from CFSR. IN.grb = time series of TMP2m (only) for the given year # create an inventory where record 1 is last wgrib2 IN.grb -for 2:: > junk.inv wgrib2 IN.grb -d 1 >> junk.inv # process the file with record 1 last wgrib2 -i location1.txt wgrib2 IN.grb -lon -80 40 > location2.txt Trick 1, extract location1 and location2 at the same time (2x savings) Code that reads location.txt needs to be changed wgrib2 IN.grb -lon -90 30 -lon -80 40 > locations.txt Trick 2, only decode the fields that you need (ex. 2 m temp, accumulated precip) 2 m temp = "TMP:2 m above ground" accumulated precip = "APCP" wgrib2 IN.grb -match "(:TMP:2 m above ground:|:APCP:)" -lon -90 30 -lon -80 40 > locations.txt notes: -match "(A|B)" means only process fields whose inventory matches A or B. The argument of -match is a regular expression (unix speak) which is expressive. In our example, only two fields need to be decoded as compared to the 2x300 fields. Comment: grib2 supports irregular grids (grids defined by longitude and latitudes). To save the values for 10 locations as a grib2 files, try wgrib2 IN.grb -irr_grid "lon1:lat1:lon2:lat2 .. lon10:lat10" RADIUS OUT.grb RADIUS: the nearest neighbor has to be within RADIUS km to be used. the nearest neighbor is used. storing the grib messages as submessages will make the file smaller. wgrib2 OUT.grb -tosubmsg SMALL_OUT.grb (35) I needed to convert a direct access big-endian IEEE time series of daily 00Z Z500 fields for 1980. step 1: make a template file, a grib2 file with 1 record with the right grid (same as IEEE file) and metadata (except time and level). Here is the result bash-3.2$ wgrib2 template 1:0:d=2012010100:HGT:500 mb:anl: The grid matches too. bash-3.2$ wgrib2 template -grid 1:0:grid_template=0:winds(N/S): lat-lon grid:(144 x 73) units 1e-06 input WE:NS output WE:SN res 48 lat 90.000000 to -90.000000 by 2.500000 lon 0.000000 to 357.500000 by 2.500000 #points=10512 here's the bash command to convert the file into grib2. i=1 while [ $i -le 366 ] ; do wgrib2 template i=`expr $i + 1` done | wgrib2 -i template -set_ts_dates 1980010100 1day 1 -set_lev "700 mb" -no_header \ -big_endian -import_ieee $ieee -set_grib_type c3 -grib_out Z700.grb2 How many lines of fortran was saved?