How to use Transform-related mapping codes

Contents

  1. Introduction
  2. Shadow
  3. MergeBitmaps
  4. ShadePalette
  5. Going to Transform or NCSA codes
  6. A sample run

Introduction
These codes allow for the
  1. Shadowing of any integer*2 bitmap using the same algorithm used by the USGS in making their shaded relief maps (see USGS Miscellaneous Investigations map I-2206)
  2. Combining any two bitmaps (integer or real data) into a single 8-bit value at each point, where the top n bits are from one file and the bottom (8-n) bits are from the other. This can be used to make shaded-relief maps in things like Fortner Research's Transform.

Although these codes have been written and compiled on a Macintosh, the source code is fairly standard Fortran save for some Open statements and could be ported easily to other systems. (Though some caution is warranted--some systems have interesting i/o characteristics to binary files). So the Mac interface isn't really well used, but given the filterish nature of these codes, this isn't a serious drawback.

Shadow
This is to make "shaded relief" maps of a 2-byte integer bitmap. This uses the algorithm used by the USGS in making their shaded relief maps and avoids the less desireable use of simple derivatives often employed. It is primarily intended for use with TopoToAlbersFull output, but will work with any appropriate file. Simple modifications would allow the use of any format of data.

The basic idea is that you specify a source illumination (sun angle) and the input elevation file and the program generates an output binary file of brightnesses of each pixel (the exact tricks are discussed below). The output file is a binary file of IEEE format real (4 byte) values (floating point numbers) ranging from 0 to some value less than 1. (All slopes considered to be shaded are forced to be 0). This same algorithm can be used within Transform, but this code is usually more convinient (and allows for larger files to be made).

You are asked for the input file and then asked to tell the code how long each input line is (namely, how many pixels wide is this image). If you saved the height x width information in the file name from TopoToAlbersFull, this is simple--use the second number (it is the same value you gave TopoToAlbersFull). The code then figures out how many lines (i.e., how high the bitmap is) and outputs this for your verification. If the output is not an integer, you may have input the wrong value for the width of the bitmap. You then name the output file and provide the illumination parameters.

The illumination parameters are mostly simple. The first number is most confusing; it is the ratio of the distance between grid points and integral increments of the function. This allows the use of non-topographic inputs. For output from TopoToAlbersFull, you should have seen an estimate of the grid spacing in meters; this is the number you want to input here. It need not be exact; in fact, you could be quite far off and still produce an acceptable looking image. Next you give the azimuth and elevation of the sun, where the azimuth is the direction, in degrees clockwise from "up" on the bitmap that the sun (source of illumination) is. In general, this should be to the left or upwards (even though this doesn't often occur on the Earth!) as this produces images usually recognizable to most people. Using illumination azimuths of 0, 90, 180, or 270 degrees runs the risk of highlighting grid edges from some databases (though the use of a projection in TopoToAlbersFull reduces this problem); if you see strange lines running across your output, you might change the sun azimuth. The sun elevation is in degrees upward from "the horizon" (flat). Moderately low values (20-30 degrees) seem to produce an acceptable amount of shading. Having the sun overhead (90 degrees) will produce a very flat image. The final value is the vertical exaggeration; you could recover from a bad estimate of the ratio here. This tends to exaggerate the difference between steep slopes of different orientations. Values of 1-2 seem to work fine.

You then sit and wait awhile (depending mainly on the speed of your hard drive) for the code to finish.

Details. What happens in this code is that the orientation of the bitmap surface is calculated for a series of triangles, where the vertices are the point in question, the point to the right and the point below. Purists will note that this isn't really the most desireable way to figure slopes, but it is (1) fast and (2) preserves high resolution variations in slope. Because of the biases, if you were being finicky about using the output file, the output grid is really down and to the right of the input grid by about 1/3 to 1/2 of a grid spacing. At the right edge, the three points are the point, the point to the left, and the point below. At the bottom, the three points are the point, the point to the right, and the point above. Again, this is not technically legal, particularly as it means that the edges have a different offset from the original grid than the main body of the output bitmap, but in practice the results look ok. The output value (illumination) is given by the equation I = 1/(1+[cos e / cos i]), where e is the angle between observer and slope normal and i is the angle between the illumination source and the slope normal. The rest is simply algebra.

MergeBitmaps
This is a rather specific code for use with Transform or NCSA software that uses 8-bit colormaps to illuminate arrays of data (which is what these bitmaps are). By using a rather simple trick, one can make decent looking "shaded elevation" maps using these codes. The trick is to basically store one variable (e.g., elevation) in some of those 8 bits and another (usually shaded relief, as from Shadow) in the remaining bits. With some clever work on the colormap used by Transform or the equivalent NCSA codes, you can make shaded elevation maps, where the colors represent the elevation and the brightness reflects the shading. An extra bonus of mastering this technique is that you can impose relief on other (preferably smoother) functions (say, Bouguer gravity anomalies) to produce a combined map that can, in some instances, be quite enlightening. If you aren't careful, though, you might just have a muddled mess.

To begin with, you must have two files with the same number of points; they need not be the same format. The files from the codes above will generally be a topography file, which has 2- byte integers, and a shaded relief file, which has 4-byte real (float) values. After launching the program, you first select file 1. This file will be put in the higher bits of the output file; for the colormaps I use (and included in this package) this variable will be the one colored. You are then asked for the type of data in this file, where the choices are short integers (2-byte integers) (Int*2), long integers (4 byte integers) (Int*4), floats (IEEE 4 byte floating point numbers) (Real*4), and long floats (IEEE 8 byte floating point numbers) (Real*8). You next enter the length of the line (=width of the bitmap), which once again is the same value used in both TopoToAlbersFull and Shadow. You are then told how many records are in the file (i.e., how high the bitmap is); if this is not an integer, you might have entered either the wrong type for this file or the wrong length of each line.

You now repeat the same steps for the second file, which will go into the lower bits of the output file. If the lengths of this file and file 1 disagree, you will be questioned; if you continue, the shorter length of the two files will be used.

Now you specify how the two files are to be combined. First you indicate how many bits you wish to use for the first array (=file 1). Larger numbers allow for more gradations of this first file at the expense of the second file. It's all a matter of taste; 4 is a decent choice to start with. The code next tells you how it will bin the data in file 1, indicating the bin size and the base value (which is the minimum of the file). The defaults will include all the data in the file. You might not wish this if the datafile includes "not real data" values from TopoToAlbersFull and so you might specify a new minimum value and bin size. Data values that fall outside the range specified will be grouped with the closest possible values. You are allowed the same options for file 2, and then you designate the output file. The output file will be a binary file of 2-byte integers organized in the same manner as the input files. (note that space could be saved if this were output as single byte values). The output values will range from 1 to 254 (values of 0 and 255 are omitted to assure Transform of having black and white in the colormap).

An an example, imagine that file 1 has a value of 12 and file 2 a value of 0.4. You selected 5 for the number of bits for file 1, it has a minimum value of 0 and a bin size of 5. File 2 has a minimum of 0 and a binsize of 0.3. Then the output value for this point will have "2" in the left 5 bits (= 2 * 8) and "1" in the right 3 bits for a total value of 17. Other points with the same value of 12 in file 1 will range from 16 to 23 (as will the other points in that bin, namely values of 10.00+ to 15.0).

Although only the low byte is used, the output file is a binary file of 2-byte integers.

MergeHDFBitmaps is similar but uses 2 HDF files (only the first dataset in each) as inputs.

ShadePalette
This code will create a version of an existing continuous color palette for use with the shaded- relief output from MergeBitmaps. First save an existing palette from View in raw format. Then run ShadePalette, opening the saved palette. You will be asked for the number of high-division bits (the number of bits assigned the first array in MergeBitmaps). You then input the range of values from the original palette to use (1, 254 for using the full palette; you can use subsets with this command). You are shown the lightness range from the file and are asked what the minimum and maximum lightnesses should be (0 is black, 100 is white). If lightnesses in the existing palette exceed your bounds, they are reset to a point 10% from the edge of your input bounds. Finally you are asked how you want your shadow gradients calculated. If linear, the lightness will vary linearly from the central value from the original palette to the max and min you input. If quadratic, lightnesses vary with the square of the shading. A value in between will produce a linear combination of the two effects; something around 0.25-0.5 works pretty well.

The output file is in raw palette format and can be loaded by Transform or View; see the discussion below for the exact strategy for viewing shaded images.

(The raw palette format is a string of 256x3=768 bytes with the three sets of 256 bytes (unsigned single-byte integers) being the red (1st 256), green (2nd 256) and blue (3rd 256) values for each palette element. The first and 256th element are usually reserved to be white and black.)

Application to Transform and equivalent NCSA packages
The outputs from all of the above codes can be used with the Transform package (or the equivalent NCSA public-domain packages (only UNIX seems supported now); discussion below is for Transform but probably is very similar to how you'd use this in the NCSA codes, given their common heritage).

To input: Select "Open..." and select the desired file. You are then asked what file type this is (binary array) and some particulars about the array (the number type is 2-byte (16-bit) signed integer (signed short) for outputs from TopoToAlbersFull and MergeBitmaps and is 32-bit (4-byte) IEEE floating point for Shadow output files). The array size can be determined by later version of Transform if you use the suffix (aaaxbbb) at the end of the file, as is suggested by the TopoToAlbersFull code, where aaa is the number of rows (height of bitmap) and bbb is the number of columns (width of bitmap). If you didn't use this file naming convention, you have to enter the proper values. You want to skip 0 bytes at the beginning, you don't want byte-swapping or a 3-D matrix.

You should now have a display of the numbers in your file. Generate an image, using an appropriate color table, and see how it looks. Some of the possible problems you might see include: the map being rotated or flipped (check the "When Importing..." things in the Preferences dialog--you might be transposing or flipping there), and the data being apparent garbage (probably chose the wrong data type, asked for byte swapping, gave the wrong size for the number of rows or columns, or let the number of bytes skipped be more than zero). For best results, you should probably make sure that the aspect ratio is 1 (check the "rectangle size" either from the toolbar in the image window or from the image menu). The aspect ratio being 1 means that both axes have an equal zoom value.

Display: For files from TopoToAlbersFull (topography) the Earth or Earth-land color maps provided with Transform can produce some nice results (these might not be in your Table menu by default, in which case you can use these colormaps by opening the color map file using "Load Color Table" and searching the Color Tables folder(s) from application disks). A trick is to get the Earth color table to put the green/blue contact at sea level; by experimentation I know that if you reset the limits of the data to -9999 and +7446, you will get sea level at the right spot. Output from Shadow looks fine with the grayscale map.

Output from CombineBitmaps is trickier, because none of the standard colormaps are made for this peculiar combination (though ShadePalette will do this). I have made several color tables (which should be with this file) for use in some possible situations. These are editted versions of the Earth-Land and Rainbow color tables and generally indicate the bit assignments within the name (3x5 bit indicates that the colored variable has 3 bits assigned it and the black-to-white variable has 5 bits) and whether the black-to-white gradients go from solid black to solid white ("linear") or from a very dark color to a very light color ("subdued"). You must also make the limits of the data 1 and 254 (the Min and Max of the second tool in the image window or af an associated Color Bar); it is quite possible that your data do not extend to these values, and the slight offset can make the image appear bizarre. Note that the CombineBitmaps trick does not allow for interpolated images or resampling of data.

Getting the right color table: If you don't like the way things look, you may have to edit the color table. The View utility that ships with Transform lets you do this, starting from any color map that is available (but you have to open an image first within the code to make the color editor option active). View is also where you can add color tables in older Spyglass products (this got broken in Fortner's upgrade of View to 3.0.2-- I know of no way to add to me color tables in Transform since then). The fastest way to make a color table (palette) for CombineBitmaps output is with ShadePalette program. You can look at the color tables used with the CombineBitmaps outputs to get an idea how this trick works and develop your own using the Color Table Editor in View.

A sample run
The lines below contain the screen output from successive runs of TopoToAlbersFull, Shadow, and MergeBitmaps to illustrate the use of these codes. User input is preceeded here by a ">" and comments are bracketed by "< >". Files mentioned are in the archive release of the codes.

TopoToAlbersFull: Enter Projection Parameters: Standard Parallels: >29.5,45.5 Lat, Lon of origin >29,-118 Use ETOPO5 (global) (E) or TOPO30 (contiguous US) (T)? >T Fill only within entire lat-lon range (L) or fill entire bitmap rectangle (within corners specified) (R)? >R Lat and Lon of one corner: >36,-119 Lat and Lon of opposite corner: >37,-117 <Next few lines is different if "entire lat-lon range" is chosen> Other corners of output map are 36.99989 -119.0134 and 36.00011 -117.0132 Maximum lats and lons on this plot will be Latitude: 36.00000 37.00414 Longitude: -119.0134 -117.0000 <X and Y ranges are in meters; you can use this to anticipate your output scale> X range: -89339.30 88160.63 Y range: 780651.2 892660.6 Desired number of x points: >400 Pixel spacing about 443.749843750000 m # y points = 252 <lines below give the number of points in the database/output pixel; when> <these are less than 1, interpolation might be warranted; when greater than 1> <averaging of points might be warranted> Points/pixel : x y lower left: 0.5901283 0.4807087 lower right: 0.5901283 0.4807087 upper right: 0.6097993 0.4700536 upper left: 0.5966853 0.4802989 bottom central meridian: 0.5966853 0.4757910 top central meridian: 0.6048815 0.4753811 Overall smallest : 0.5901283 0.4700536 Do you wish to interpolate between points? >y <User is prompted for output file name; code spends quite some time doing the interpolations <and i/o; this code will run in the background> <Output from this test was called "So Sierra Topo (0252x0400)" and an image of it> <(So Sierra Topo.PICT) is in the examples folder> -------------------- Shadow: Input file (bitmap): <dialog box appears and you select the input file; the name of the file is echoed to the screen> So Sierra Topo (0252x0400) How long is each line? >400 Then there are 252.0000 records OK? >y <reconsider here if the number of records is not an integer!> Output file name: <dialog box appears and you name the output file> Approximate ratio of grid spacing to function: e.g., for topo in meters at a spacing of 2.5 km, use 2500 <this should be about the "Pixel spacing" number from TopoToAlbersFull> >400 azimuth and elevation of sun >330,25 Vertical exaggeration: >2 <Some waiting as calculations are done> <Output file here is "So Sierra Sha (252x400)" and the image is "So Sierra Sha.PICT" in the> <examples folder> ------------------------- MergeBitmaps Input file 1 (bitmap) (will be higher bits): <User given a dialog box to select file; name is echoed to screen> So Sierra Topo (0252x0400) File 1 type S for Int*2, L for Int*4, R for Real*4, D for Real*8 >S How long is each line? >400 Then there are 252.0000 records OK? >y File 2 type S for Int*2, L for Int*4, R for Real*4, D for Real*8 >R Input file 2 (bitmap) (lower bits): <dialog for selecting file 2 presented> <if the lengths of files 1 and 2 differ, notification will be here> Getting limits... Min, max of first file: -33.00000 4072.000 Number of bits for first array (2 to 6) >4 Number of integer array levels= 16 Default binsize is 256.5625 from a base of -33.00000 Enter minimum and bin size for file 1 (0 bin size for defaults) >0,0 Min, max of second file: 0.0000000 0.6498090 Default binsize is 4.0613063E-02 from a base of 0.0000000 Enter minimum and bin size for file 2 (0 bin size for defaults) >0,0 Output file name: <Output file from this test is "So Sierra Merge (252x400)" and the PICT file is> <"So Sierra Merge.PICT">

Return to Overview of Shaded Topography codes

Return to C.H. Jones home page

Craig Jones, cjones@mantle.colorado.edu