Section I of this Guide is for those who are unfamiliar with IDL or who are trying to decide whether to adopt it. Sections II and III are for users. Although the examples are taken from applications in astronomy, most of the Guide is general enough to be useful to workers in medical imaging, geophysics, or other areas for which IDL is well suited.
The Interactive Data Language (IDL) is a proprietary software system distributed by Exelis Visual Information Solutions, Inc. (http://www.exelisvis.com), originally Research Systems, Inc. IDL grew out of programs written for analysis of data from NASA missions such as Mariner and the International Ultraviolet Explorer. It is therefore oriented toward use by scientists and engineers in the analysis of one-, two-, or three-dimensional data sets. Exelis claims over 150,000 users.
IDL is currently available in LINUX, UNIX/Solaris, Windows, and Macintosh versions. IDL device drivers are available for most standard hardware (terminals, image displays, printers) for interactive display of image or graphics data.
IDL is not simply a package of task-oriented routines in the style of astronomical software systems such as IRAF or CIAO. Instead, it is genuinely a computer language, readily understandable by any computer-literate user. It offers all the power, versatility, and programmability of high level languages like FORTRAN and C. But it incorporates three special capabilities that are essential for modern data analysis:
Users who are conversant with FORTRAN, C, C++, or other high level languages will have little trouble understanding IDL. Its syntax and operation are clear, sensible, and convenient (most similar to FORTRAN's). Because it is interactive, learning IDL through on-line trial-and-error is rapid.
IDL provides the scientist better understanding of and control over computations and data analysis by virtue of a large number of special features:
Eight versions of IDL have appeared to date. Version 8.1 is the latest.
What role does IDL play in the context of other software readily available to astronomers?
FORTRAN, C, or C++ cannot satisfactorily serve the need of individual users for interactive data analysis because they do not provide a standard interactive environment with supplied graphics device-drivers.
The data reduction and display software with which most astronomers are familiar, including IRAF, STSDAS, AIPS, AIPS++, CIAO, MIDAS, and SUPERMONGO, consists primarily of collections of specialized, task-oriented routines. They are interactive and graphics-oriented, but they operate only in a pre-compiled form that offers the user little opportunity for upgrade or customization. Constituent routines function like "black boxes" and do not provide the user with easily understandable access to their inner workings. These packages can be executed from scripts, but they are not intended as the basis of user-written applications. They are complex enough that professional programmers are required to maintain and enhance them. They adjust only slowly to user requirements. By contrast, not only does IDL offer greater transparency, versatility and efficiency, but also the whole computer-literate user community may be drawn upon to extend and improve IDL applications software.
Comparison with other popular systems:
The intrinsic capabilities of IDL coupled with its extensive user code libraries greatly enhance scientist efficiency.
The built-in journal-keeping and command recall/edit features of IDL are so important to efficient and reliable data analysis that it is something of a mystery why most other astronomical software packages do not offer them.
journal and command recall/edit facilities is a tremendously more efficient way of writing reliable code.
Because of IDL's versatility, transportability, and ease of use, there is a large, user-written, public library of interactive IDL software, now over 10,000 programs.
Many astronomy-oriented IDL routines and packages are in the public domain.
Other packages have been written by groups associated with the Sloan Digital Sky Survey, Chandra, ROSAT, COBE, Fermi, GRO, HST/NICMOS, HST/ACS, SOHO, ISO, the Spitzer Space Telescope, the Green Bank (Radio) Telescope, the Owens Valley Radio Observatory, the Submillimeter Array, Los Alamos, Lawrence Livermore, and NRL, among others. There is also a large body of related remote-sensing, medical, geophysics, and environmental research IDL software. These packages contain many programs of general interest (e.g. statistical analysis, databasing) as well as specialized functions.
Unlike IRAF, STSDAS, and AIPS, there is no central distribution point for "authorized" IDL software other than the Exelis-supplied intrinsic IDL language. However, neither does one have to depend on a small cadre of expert programmers to fix bugs, explain program operation, or provide new capabilities.
By virtue of publicly-available applications that cover the EM spectrum from gamma-ray to radio wavelengths, IDL has become the nearest thing to a "universal" astronomical data analysis system.
The IDL source code for applications packages is automatically available, so the user can use the routines as written or, alternatively, readily modify their components (as one would FORTRAN subroutines) to customize them.
IDL is ideally suited for software exchange over the Web. Because no pre-compilation is required, installation of a new package, for instance, is simply a matter of putting the ASCII source files in your IDL path.
IDL's cross-platform design ensures that you can confidently transfer code between environments (e.g. from a Solaris desktop to a Macintosh laptop).
C or FORTRAN programs can be executed from within IDL and data exchanged between the systems.
For examples of IDL computational and graphical applications, run the idldemo demonstration from the LINUX command line.
Once you start IDL and begin typing at your console, you are communicating with the main level of an arbitrarily large computer program over which you have (nearly) complete control.
People who are familiar with IRAF, CIAO, or AIPS will immediately recognize major new features in the IDL environment.
This has the additional important advantages that IDL programmers do not have to worry about file formatting of the results of intermediate computations or consume mass memory to store them.
Even novices will appreciate the ease with which accelerator commands or scripts can be fashioned to speed routine tasks.
IDL is oriented toward image analysis and signal processing, but by comparison to IRAF, etc. it also opens a new dimension of capability to make other kinds of important numerical calculations with great efficiency and to create graphical or tabular representations of them. In this regard, it duplicates much of the functionality of MATHEMATICA, MAPLE and MATLAB (though without symbolic manipulation).
A great variety of such problems, for which (if they undertook a careful computation at all) many astronomers would have written stand-alone FORTRAN or C programs, can be handled with beautiful efficiency by IDL and immediately incorporated with associated data analysis tasks.
Basic IDL functionality has not changed for the last several major releases. Instead, additions have involved areas like GUI support and object-oriented programming, which are directed more at software developers than users interested in data analysis.
IDL has many virtues, but what are its limitations?
An obvious limitation, and a significant barrier for some people, is that IDL is a proprietary system, which means that each site must purchase an IDL license. Some astronomers object on principle to paying for software.
My own view is that since the most expensive element of any such system is always the scientist labor involved, the dollar investment in IDL is clearly justified. Current prices are comparable to 1.5 months of a postdoc's salary, and for that you get access to perhaps 1000 person-years of good, astronomically-oriented software. One would have to have a very good postdoc not to consider IDL a bargain.
IDL is an interpreted rather than a compiled language. This means that large IDL programs can execute less rapidly than equivalent compiled programs written in FORTRAN or C.
This environment places a premium on software versatility, convenience, transparency, modularity, and portability---exactly where IDL excels.
The inherent vectorization of IDL's built-in software will often compensate for its interpreted structure in competition with FORTRAN or C programs that have not been properly optimized by their authors.
Because of its optimization for interactive computing, IDL is not the system of choice for large scale numerical computations (e.g. hydrodynamical or N-body simulations). However, it is valuable as a medium to explore new computational approaches in a smaller setting where raw speed is not important, and it is also excellent for visualizing, analyzing, editing, and displaying numerical data sets generated by simulation software.
A problem for novices is that help with IDL may be hard to come by at a new installation. There are, however, consultants available at Exelis, Web-based advice sites, large amounts of on-line or printed documentation, and the examples of thousands of working IDL routines in the public libraries that can help solve many software difficulties.
It is the data analysis phase, as opposed to the reduction phase, where the versatility of IDL is most useful. The IDL Astronomy Users Library supports the standard data formats output by the large astronomical reduction packages (including FITS image and table files).
If you are a young observational astronomer, you have essentially no choice but to become familiar with IRAF/STSDAS (UVOIR), AIPS (radio), CIAO (X-ray), etc. because these provide the primary standardized data reduction procedures in your field. The main decision, therefore, is what other systems are most profitable to learn. IDL is a prime candidate.
Finally, the rapidly proliferating set of IDL applications routines is both a strength and a weakness. While one has access to a wide variety of useful software, this is not always fully tested since the authors typically apply it to problems of limited scope. The hardest part of using IDL often is determining what routines are available for a given application and deciding which is best to use.
Overall, in exchange for its improved versatility and power, IDL requires a higher level of computer skill than do systems like IRAF, AIPS, or CIAO.
On balance, IDL is an invaluable tool for most observational or theoretical astronomers.
Resources: For lists of important astronomy-oriented IDL resources (guides, help files, demos, tutorials, software) other than those packaged with the Exelis distribution, see:
Demonstrations: To get a feel for what IDL can do, try running the package of standard IDL demos supplied by Exelis. Start from the LINUX prompt (within an X-window) and type idldemo .
Tutorials: A set of three introductory IDL exercises which introduce its basic features is available on the UVa ASTR 511 home page. Other sites offering IDL tutorials are linked to the Astronomy User's Library.
Software List: Six different levels of IDL code will be useful to you:
here. (Note: the STScI MAST archive distributes an older version of MOUSSE, ca. 1993, from http://archive.stsci.edu/uit/analysis.html. That has been superseded by the versions in the TAR file.)
Some MOUSSE routines employ special common blocks to track image displays, etc., so MOUSSE requires that a special "startup" file be executed (see below). MOUSSE routines are self-documented. Older routines have not been updated since 1996 and may exhibit incompatibilities with more recent IDL releases, new data formats, and so forth. Let me know if you encounter such difficulties. At UVa, Mousse software is stored in /astro/idl/Mousse .
Acknowledgements: In any published work utilizing IDL software written by others, you should acknowledge the author. If you use the Goddard software, the GSFC Astronomy User's Library group and curator Wayne Landsman should be acknowledged. Modified versions of others' software should propagate the authorship list in the header section of each routine.
This section provides an introduction to intrinsic IDL and user-supplied applications routines frequently used in 2-D image processing and related kinds of data evaluation and computation. Only the most common options for each command are listed.
This guide is oriented toward the UVa/LINUX installation of IDL. However, I have tried to clearly distinguish details which are specific to the local system so that others will be able to use the guide.
To configure your session on the main UVa Astronomy LINUX server, you must first source the standard setup file /astro/idl/idl_env.csh . You can do this manually or modify your .cshrc file to do it automatically. This will set you up to run the latest version of IDL with proper links to all current software directories, including the Astronomy User's Library, MOUSSE, and ATV.
In editing, be careful not to give the ^d command on an empty line, since this will terminate your session!
find=where(x gt 100, count) & print,count .
--> A set of & -linked commands on a single line constitute a "micro-program" which can be executed, then edited and re-executed with a couple of keystrokes.
Exelis Documentation
Using IDL, The IDL HandiGuide/Quick Reference, Building IDL Applications/Application Programming,
The IDL Reference Guide
. These are accessible on-line from within an IDL session through a Hyperhelp facility. (Note: the titles of some manuals have changed with version number.)
manual for V3 and V4 take you through a number of interesting sample applications and provide helpful hints.
The best way to remind yourself quickly of the operation of intrinsic IDL routines is to refer to the IDL Quick Reference. Unfortunately, this "quick" guide has become rather unwieldy, and, at least for a reference on basic IDL functionality, it may be easier to use one of the older versions of the IDL HandiGuide.
You will probably find it helpful to make a hardcopy of the
sections titled "Functional List," "Syntax," "Statements," "Executive Commands,", "Special Characters," "Subscripts," "Operators," "System Variables," and "Graphics Information." That will total only about 30 pages.
Browser Access to Documentation:
Determining IDL Version Number
Learn by Example
Informational Commands
if (n_params(0) eq 0) then begin print,'CALLING SEQUENCE: program_name,var1,var2,var3' return endif
man : The man command without arguments produces a graphics menu display similar to that generated by ? but which provides information on AstUseLib, MOUSSE, and other user-written routines. man is MOUSSE program written by Wayne Landsman and is available at: http://idlastro.gsfc.nasa.gov/ftp/contrib/landsman/help/.
The man function operates by finding the first file with the name "[command].pro" in the IDL_PATH and listing all those lines in the file between the two lines starting with ";+" and ";-" in the first two columns. It is strongly recommended that you place headers of this type in your own *.pro routines. There is a standard internal header format adopted by most IDL coders, though you do not have to follow this in your own routines.
Unfortunately, intrinsic IDL routines cannot be accessed by man .
doc_library,[command-name-string] has similar functionality; it also lists the directory path to the file.
Note: one deficiency of current user-supplied procedure documentation is that the dependencies (routines called by a given procedure) are not listed. If you use getpro to obtain and edit a supplied routine to behave differently, this may have unintended consequences for the functioning of other procedures. One method of looking for such interactions would be to grep the ASCII files in the library directories for other references to the routine you intend to change. A quick way to identify what subroutines are called by a given program is to .run the program as the first step in an IDL session. Each subroutine is listed on the screen as it is compiled.
Use the printf command to send the output to a selected file or other external unit.
help,/pro and help,/func : lists names and parameters for compiled procedures or functions, respectively (other than intrinsic IDL routines)
tvstatus : a MOUSSE routine that lists properties of active windows
Your journal file will be written and closed when you type journal again or exit your IDL session.
It is hard to exaggerate the value of keeping IDL journal files for serious work. It is good practice always to use a journal file so that you can check or reproduce what you have done, recover errors, and so forth. If you edit out the blunders and undesirable clutter of your journal files, you will have a handy running record of your work.
Journal files permit rapid re-creation of an IDL session, in whole or part, or (if the activity was worth saving and repeating) can be edited into the form of a main program, subroutine, or script.
Warning: the journal routine will overwrite an existing file with the given [filename] without warning. Use unique names for each session. Also, the journal file may be lost in the case of an IDL crash or a system failure (e.g. power outage). For long sessions, you may want to plan for more than one journal file.
All intrinsic IDL programs are compiled and ready for execution when you begin your session. Other programs are normally compiled only when you request them. A list of all compiled non-intrinsic routines is presented if you type help,/rou .
NOTE: all IDL main programs, subroutines ("procedures"), and functions are assumed to be in files with the explicit extension '.pro'.
testdata = fltarr(512,256) creates a 512x256 2-D floating point array with zero entries.
Note also that (again to conform to graphical standards) the first element of any IDL vector or array is always at index 0 not 1. For example, in the array z=[20,30,40,50] , z[0]=20 , and z[3]=50 . But z[4] is undefined.
testdata = findgen(512,256) creates a 512x256 2-D floating point array with each element set to the value of its 1-D subscript (starting at 0). Thus, testdata(100,0) = 100.0 while testdata(0,100) = 51200.0.
name = "Maxwell" creates a string
"Structures" in IDL are sets of other variables (an arbitrary mixture) that can be referred to by a single name. They can be very useful in data analysis problems.
The help command gives you a listing of active variables and their properties.
Building IDL Applications/Application Programming
for a complete description.
IDL provides over 750 intrinsic mathematical and other functions, not to mention the hundreds of others present in the IDL User's Library. For example:
IDL Reference Guide
Many of the operators and mathematical functions take vector or array arguments as well as scalars.
Typical IDL command lines might look like this:
a = [45, 90, 135, 180]*(!pi/180.) & y = sin(a) z = ((x-xc)^2)/(2.0*sigma^2) & gaussfun = exp(-z)/(sigma*sqrt(2.0*!pi)) sm_im = smooth(im,5) if (photon_count lt 30) then ran_count = randomn(seed,poisson=photon_count)
Program files can contain more than one IDL program. Simply concatenate programs together. All will be compiled by the .run command and recognized separately in future calls. However, it is not recommended that you pack programs this way. Among other things, you lose the capability of reading internal headers with the man command (see above).
You do not have to give a separate .run command in order to execute a subroutine which is in your path and has a file name identical with its procedure name except for the added '.pro' extension. Simply type the command's name, with all required parameters, and IDL will automatically search the path for such a file and execute it. If the file name does not match the procedure name it contains, then you must .run the file separately.
You do have to use a .run [name] command to execute a main program the first time. To re-execute, you can use .go .
The .run [name] command will compile but not execute a procedure or function file.
You must re- run any command whose procedure file you have edited during your session or the revisions will not take effect.
.con : Continue a program interrupted by a pre-programmed stop command.
.go : Restart a previously compiled main program from beginning.
@[name] : Execute the "batch" file [name].pro containing a list of IDL commands in the same form as they would be entered from the keyboard. Can be used to run a large background job or a user-specific initialization file at the beginning of a session. Batch files are frequently used to duplicate or iterate a set of commands derived from an earlier IDL session (e.g. for graphics output). Such a set is called a "script".
You can set up common blocks, compile procedures, and so forth with batch files. Text after a semicolon ( ; ) on a given line is ignored by the compiler.
An efficient way to update or execute parts of a script---e.g. a command sequence captured using the journal utility---is to run an editor on the script in one window while executing IDL in another and cutting-and-pasting snatches of code between them. This method is especially useful when you have a large number of options (e.g. doing statistics, histograms, and plots on a data file containing many different variables).
execute : The execute command executes any string containing a proper IDL command. The syntax is
Here result will be 1 if the command was successful and 0 otherwise.
This offers a powerful means of adapting program operation to contingencies during execution. For instance, you can create a new variable whose name and characteristics are based on information read from a file with contents unknown ahead of time. Use built-in string manipulation utilities to create the command string.
Programs are structured text files of IDL commands. They can be stored anywhere in your IDL path. There are three different types of programs, each of which interacts slightly differently with your active IDL session. Coding rules differ slightly in each case.
Main programs cannot be executed by a procedure or function.
PROCEDURE_NAME,input_parm_1,input_parm_2. output_parm_1,$ output_parm_2. keyword_1=value1,keyword_2=value2.
Alternatively, in the case of switches, keywords can be set to a value of 1 by using the following syntax:
The keyword name in a calling statement can be truncated to as short a string as allows the keyword to be uniquely identified.
The keyword_set(name) function can be used within a subroutine to sense whether a particular keyword was set by the user in the current call to the subroutine.
IDL supports "keyword inheritance" such that keywords which may not actually be defined in the procedure declaration but which are included in the command that called the procedure can be passed through to other procedures that do recognize them. This adds considerable flexibility to writing interactive software. See the IDL manuals for details.
PRO Name,parm_1,parm_2. keyword_1=keyword_1.
and must close with a return and an end statement.
result = FUNCTION_NAME(parm_1,parm_2. keyword_1=keyword_1. )
and must close with a return,[output] and an end statement.
To illustrate some of the differences between main programs, procedures, and functions, here are three corresponding versions of the code for determining the length of the first dimension of a given variable (scalar or array) using the built-in SIZE function. Each version is assumed to be stored in a file named get_dim1.pro in your IDL path.
; MAIN PROGRAM: get_dim1.pro s = size(a_in) if (s[0] eq 0) then d1 = 0 else d1 = s[1] end
PRO get_dim1,a_in,d1 s = size(a_in) if (s[0] eq 0) then d1 = 0 else d1 = s[1] return end
FUNCTION get_dim1,a_in s = size(a_in) if (s[0] eq 0) then d1 = 0 else d1 = s[1] return,d1 end
.run get_dim1 print,d1
get_dim1,a_in,d1 print,d1
print,get_dim1(a_in)
An alternative is to set on_error,1 at the beginning of a session, which automatically returns to the MAIN level after an error. on_error,0 sets the default mode (stay in the subroutine).
You can check where you are by typing help . The first line of the printed information block lists your current location (procedure name and line).
Importing software
You can do this before or during an interactive IDL session (though you cannot change your IDL path during a session). You must, of course, be sure you have all the subsidiary programs needed by the new software also in your path.
Data transfer to memory
Note that this is in contrast to IRAF and most other standard astronomy software packages, where there is no intermediate data storage, all manipulation begins and ends with data stored as files, and one must refer to data sets by their file names at all stages in the analysis process.
IDL can read/write files up to 2.2 GB in size (and longer on some platforms). However, your host computer may have limitations that prevent access to files this large. See the "Files and I/O" chapter of the
Building IDL Applications/Application Programming
manual for information on handling large files.
Note that any filename must be given as a string in the commands described below. That is, it must be quoted (unless you use a pre-defined string variable). Example: fits_info,'m87_nucleus.fits (the trailing quote mark can be omitted if it is the last entry on a line). Wildcard notation can be used with routines that can take multiple file input (e.g. fits_info,'m87*.fits ).
Directory Routines
The basic IDL routine for reading ASCII files is readf . It is normally used with the format keyword, which specifies the structure of the file.
get_lun,unit openw,unit,'InputFile' form='(a15,3x,f9.5,3x,f9.5,3x,f6.2)' for i=0,numtarg-1 do readf,unit,format=form,$ targid[i],radeg[i],decdeg[i],vmag[i] close,unit
In this approach, the number of data lines in the file ( numtarg ) must be defined ahead of time. An intrinsic IDL utility that can help with this task for ASCII files is file_lines .
Because the for statement in this script refers to individual elements in the arrays designated to hold the incoming data, these arrays must be pre-defined to accommodate the number of elements expected to be in the InputFile . E.g. radeg=fltarr(numtarg) . The arrays can also be defined to be larger than entries in any likely input file and then truncated after the script executes.
An alternative approach, which does not require knowledge of the file length, is to read the file line by line, checking for the "end of file" mark, and then bumping up each vector after each successful read: e.g. targid = [targid,newid] , where newid is the latest value read.
Depending on the structure of the input file, readf can also be used in a free-format mode, where the format keyword is not needed and file elements are separated by commas or white space. See the IDL manuals.
The columns can be separated by blanks or commas (or other characters specified by the optional delimiter keyword) but do not have to be aligned.
The vector variables into which the columns are read by readcol do not have to be predefined.
The open and close commands are included within readcol , so no separate commands are needed.
readcol accepts simplified format specifications and can read alphanumeric fields as well as numeric fields (however, no blanks or commas can appear in alphanumeric fields).
readcol is well suited to reading the comma-separated-value files produced by spreadsheet programs.
readfmt is a corresponding AstUseLib utility for fixed-format entries. You must specify the format, but the routine will execute much faster because it does not have to test the structure of each input line.
Both the readcol and readfmt routines have the nice feature that they will skip over comment lines (or other lines with non-matching formats) without choking.
The basic IDL routine for reading unformatted binary files is readu .
IDL supports the creation and reading of portable binary files with the optional xdr keyword used with the open procedure. See the IDL manuals.
The file types discussed in the rest of this section are all modified binary formats which are standardized or internally documented such that they are portable among computer operating systems.
The standard data format for most astronomical data sets today is the Flexible Image Transport System (FITS) format. The IDL Astronomy Users Library contains a large package of routines to manipulate FITS files. Intrinsic IDL supports a variety of other common scientific data formats (see the manuals for details) with similar functionality.
fits_info and fits_help list information concerning the structure of a FITS file (number of extensions, sizes of headers and arrays, etc) without reading the entire file. This can be important to confirm the type of file (image, table, etc.) with which you are dealing. fits_info will operate on a list of files.
headfits (a function) will read the full header of a FITS file on disk into an IDL variable (without reading the data). You can then display the contents of the header with either the print or hprint commands (the latter intended for FITS headers). headfits will work on compressed files (*.Z or *.gz).
This command reads the image data in the FITS image file m87_nucleus.fits in the current directory into active variable m87 and the corresponding header section of the file into active string array variable hdm87. m87 will have the characteristics (byte, integer, float, etc.) of the FITS file data.
NAXIS, BITPIX, DATATYPE ), then an error message will appear. However, you can create a dummy header which does match the data in this case and store it on the image directory; see "Image Storage" below.
--> The optional noscale keyword used here prevents the (default) automatic scaling that converts the image values to fluxes using calibration parameters in the header. For many purposes, the scaling is a nuisance.
fits_read,'sources_file.fits',table,hdtable tbhelp,hdtable ; List contents of each field ; from the header ; Extract RA and DEC vectors, based on ; tbhelp listing: rad = tbget(hdtable,table,1) decd = tbget(hdtable,table,2)
ftab_help,'sources_file.fits' ; List contents ftab_ext,'sources_file.fits','RA','DEC',rad,decd
To add a history comment to a FITS header variable:
To add a new parameter in a FITS header variable or modify an existing one, use sxaddpar .
Exelis also supplies an intrinsic file reader that will read most commercially important image file types. Use the command checkim=query_image([filename],info) to determine whether a particular file is in a suitable format. If so, the returned value of checkim will be 1 (0 if not). For details on the file structure, type print,info,/str . Use the read_image function to read the file and its associated color tables, if any (see the help files and the next section for information on color tables).
Endian-Conversions
IDL includes the swap_endian function, which allows you to change the storage convention for data in your session. It also allows you to identify the convention adopted by your computer, as in the following:
a=3.0e5 ; Define a large number b=swap_endian(a,/swap_if_big_endian) & print,b ; If b is different from a, your machine is big endian ; If b is the same as a, your machine is little endian
Much of the power of interactive computing comes from your ability to make immediate displays of plots or images on your terminal. Hardware considerations impinge more on making such displays than on other aspects of interactive computing.
This section describes the underlying mechanics of image graphics and the basic IDL command-line routines for making image displays. If you are only interested in getting a quick display of astronomical images, skip to the description of the ATV display tool in the next section. For making plots, see 10. Plots below.
Color Graphics Displays
Modern computer monitors, probably including the one you are using, feature 24-bit displays. That means they are capable of utilizing all possible 3-color combinations. IDL image displays using this maximum color palette are known as true-color displays.
Naively, you would expect that anything labeled "true" has got to be better than something labeled "pseudo." And you would be right---if you were interested in processing commercial digital camera images or making glamorous press-release versions of astronomical images. But that's not the main concern of most astronomers. Instead, you will discover that working with displays in pseudo-color (native to an 8-bit monitor or emulated on a 24-bit monitor) is most appropriate and convenient.
Accordingly, most of the subsequent discussion is aimed at indexed-color/pseudo-color displays, whether on 8-bit or 24-bit monitors.
Displayed Arrays and Color Tables
In the case of a color photograph, for instance, a 24 bit monitor can display the full palette of colors that are embedded in the image, and hence 24-bit displays are called "true-color" displays. However, the colors do not necessarily have to correspond to perceived ones. The most widely used true-color image file format today is the JPEG format (though this is not normally employed for scientific data).
For a "gray-scale" color table, R[n]=G[n]=B[n] . If the three vectors contain different entries, a colored display will appear. These colors have no necessary relation to the physical appearance or properties of the object in the image; hence such displays are called "pseudo-color" displays; more generally, they are called "indexed-color" displays. However, there is a direct correspondence between the index values in the image array and the colors which appear on the screen. Although it does not add fundamental information, pseudo-color display of a 1-byte array can be very useful in exploring subtle structures in a complex image, for instance. Pseudo-color displays need not appear "unnatural"; they can closely approximate a true-color display but are inherently limited to only 256 levels of color/intensity. The GIF format stores images in 2-D indexed form with an accompanying color table.
On a 24-bit monitor, a color table is used in the "emulated" index-color mode set with the device, decomposed=0 command (see next entry). But color tables have no effect in the standard true-color mode described above.
Setting a Visual Display Mode on Your Computer
The retain=2 keyword requests IDL to handle the "backing store" for your display (which holds copies of parts of your display that have been overwritten); this avoids malfunctions that can occur if instead your windows system does the backup.
These commands must be given before any window is created during your session.
In the absence of a device command, the default assumed on a 24-bit monitor is true_color=24,decomposed=1 .
To check the current status of your graphics device, type help,/dev . (Note: the emulated pseudo-color mode is indicated by the entry "Graphics pixels: Combined" .)
Although you cannot change the visual class after its first implementation during a given session, in 24-bit mode you can toggle between device,decomposed=1 and device,decomposed=0 as desired.
"Color Flashing": If the rest of your 8-bit terminal display blinks out or changes color when you move the cursor into an IDL window, then IDL is using a "private color table" which gets applied to your whole screen when the cursor activates it. This probably means IDL is attempting to use more colors than were free in the shared color table. To ameliorate the problem, try this:
Exit IDL Restart IDL As the very first two commands, type: device,retain=2,pseudo_color=8 window,0,col=k. where k is the minimum number of color levels you think will be acceptable in your displays.
Color flashing will not occur on a 24-bit monitor even in emulated pseudo-color mode.
Transforming Images for Display
The array you send to your display must be a 2-D byte array (pseudo-color) or a 3-D byte array (true-color). The descriptions below are for intrinsic or emulated pseudo-color based on 2-D images and involve the use of color tables. There are various utilities for converting your original array (often floating-point) to the proper format and automatically displaying it with pre-defined color tables.
Alternatively, in pseudo-color you can use a linear transform of image values but manipulate the color table to achieve non-linear discrimination in the displayed image. In some situations, this may be preferable, but it is usually more complicated than simply changing the mathematical transform.
The basic IDL direct graphics routines for opening, displaying, and adjusting image display windows are window , which creates a new window and sizes it; wset , which defines the "current" window; and wshow , which reveals (or hides) the current window. wdelete deletes a window. These routines are used in conjunction with the two basic image display routines tv and tvscl .
window,9,xpos=750,ypos=50,xsize=512,ysize=512,$ title='IDL IMAGE WINDOW' wset,9 wshow
(All of the arguments in the window routine are optional.)
The basic tv and tvscl commands do not adjust the size of the current window to the size of the displayed array.
The routines chan,cdel,ctv,ctvscl described below are MOUSSE versions of the intrinsic IDL windowing/display routines. They offer important convenience features that make work with images faster (e.g. rescaling of window sizes to the image size, combination of the window set and show functions, arbitrary min/max values for the display, etc.). These routines use special common blocks, and to initialize those you must run the Mousse Startup File (see Appendix C). The descriptions below are for this set of substitutes for the intrinsic routines.
It is easy to iterate displays by recalling the command line and editing the min,max values.
Note: the intrinsic IDL routine tvscl does not accept min/max keywords and always scales between 0 and the maximum data value in the image. If that is not appropriate, you must independently clip the image before input to tvscl . Because of this, the display following the command ctvscl,image will differ from that of tvscl,image .
You can "flip" the color table using the mouse buttons during contrast .
Note that the type of linear transformation between the image value and the display value adjusted by contrast is only one of a very large set of possibly useful transformation functions. You may want to experiment with various logarithmic, powerlaw, segmented, and other types of transforms to find the best display for your image.
The "color-map adjust" feature in ATV is similar to contrast .
END OBSOLETE SECTION --> Color Table Commands
A set of 41 predefined color tables is supplied with IDL; these can be loaded with the loadct,N command, where N runs from 0 to 40. The default gray-scale is loaded by the command loadct,0 .
You can examine the correspondence between color and index using David Fanning's cindex routine.
On most 8-bit monitors, the display will automatically update whenever the color table is changed (at a loadct,N command, for instance). However, on 24-bit monitors you must reload the image in order for the change in color table to take effect (even in emulated pseudo-color mode).
A variety of tools is available for adjusting the supplied color tables or creating new ones. These include xloadct , xpalette , stretch , and David Fanning's xcolors .
A quick way to reverse the standard white-on-black color table 0 to the black-on-white sense preferred by astronomers, which is more sensitive to faint features, is the following. This would be useful to code as a procedure or script.
loadct,0 tvlct,rr,gg,bb,/get rr=reverse(rr) tvlct,rr,rr,rr
t=findgen(1000) & t[0] = 1 quiklog=alog10(t) ctvscl,quiklog(image)
Note: In IDL, the form array1(array2) returns the values of array1 in the elements specified by the values in array2 . array2 must be an integer or longword integer array.
8-bit pseudo-color: The window buffer contains a 2-D byte image, which is copied by the command cimage=tvrd() . Note the empty parentheses (if you wish to copy the whole window). In this case, cimage is the 2-D byte-scaled image output by the last call to the tv or tvscl routines. If you also save the current color table with tvlct,rr,gg,bb,/get , you can later reproduce the appearance of the display with the commands
tv,cimage tvlct,rr,gg,bb
24-bit emulated pseudo-color or true-color: The window buffer contains a 3-D image, which is copied by the command cimage=tvrd(true=N) . Here, the true keyword (values 1-3) determines over which dimension in the resulting array color will be interleaved. The appearance of the display can be reproduced with the command tv,cimage,true=N . You could make a permanent copy of the display in a JPEG file with the command write_jpeg,'copy.jpg',cimage,true=N . Because JPEG is a "lossy" format, the copy will not be exact; if you need an exact version (that could be read back into a later IDL session), you can store in TIFF format.
In 24-bit decomposed=0 mode, the color tables were used to create the 3-D display image, but you cannot "back them out" of cimage in order to recover the 2-D byte version that was created by the original tvscl command. You can, however, create a 2-D image that approximates the original version as follows: newver=color_quan(cimage,N,rr,gg,bb) . Here, rr,gg,bb represent the output color table needed for a proper display of the 2-D newver array. You can redisplay the new image or save it and the color table in GIF format.
David Fanning's tvread is a more elaborate, device-independent version of the tvrd function for 8- and 24-bit monitors and optionally writes various types of output files.
Object Graphics Image Display Software
References on Image Displays
manual. More details on image displays and use of color can be found in the on-line help system under "IDL Tutorials/IDL Display Concepts"; the examples mostly involve full-color image formats (e.g. JPEG, TIFF, PNG). The v7.0
manual covers image transformation techniques of interest more in geoscience and medicine than astronomy. For general background on specifically astronomical image processing, see T. A. Rector et al., "Image-Processing Techniques for the Creation of Presentation-Quality Astronomical Images," AJ, 133, 598, 2007; and R. Lupton et al., "Preparing Red-Green-Blue Images from CCD Data," PASP, 116, 133, 2004.
IDL Tutorials => Display Concepts => Displaying Images & working with color. -->
The tools described here are oriented toward analysis of 2-D images, but most are useful for any type of data array.
Array Manipulation:
Unsharp masking: ctvscl, a-smooth(a,k) displays the difference between the original image and a boxcar-smoothed version with a smoothing length of k pixels. This is fast, but the smoothed image includes the effects of any sharp structures (like stars). Better, though slower to execute, is ctvscl, a-median(a,k) .
Data Inspection:
part=image[100:200,150:250]
extracts a 101x101 subarray from the image. You can interactively determine the edges of an area of interest using curval or cursor .
To regrid an image to non-integer multiples of its original format, use frebin or congrid If you intend to make flux measures from the compressed or expanded images, be sure to scale the result so that flux is conserved in a given region of the original image. For example, if you use rebin to compress an MxM image to an NxN image using sample=0 , each pixel in the resulting image will contain the average of the corresponding pixels in the original, so that the total flux is a factor of k 2 smaller than in the original, where k=M/N.
tx=image ; create temporary copy (unless you don't ; mind corrupting the original array) finder=where(tx eq -666) ; create a list of relevant pixels tx[finder]=10000 ; highlight them (we assume normal tx values ; are
nup = findgen(10) clev = 25.*2^nup contour,image,levels=clev
For an example of placing a contour line plot over an image display, see the header of Liam Gumley's imdisp routine.
Experiment with display/cursor routines using test images containing "hot pixels," e.g. test=fltarr(512,512)+1000. & test[150,250]=20000. , steps, gradients, and so forth.
To create a 16-level "test pattern" of stepped, uniform 100x100 subimages with pixel values running from 0 to 240:
patt=bytarr(400,400) q=bytarr(100,100)+16 for i=0,3 do for j=0,3 do patt[i*100,j*100]=(i+4*j)*q ctv,patt
IDL's random number generators can be very useful in such applications. For instance, to add Gaussian photon noise to an array containing predicted photon counts:
b=size(array) noise=sqrt(array)*randomn(seed,b[1],b[2]) noisyarray=array+noise
(Here, values per pixel in the input array are assumed to be larger than ~30 counts. Otherwise use the poisson keyword to simulate photon noise.)
get_lun,unit openw,unit,'OutputFile' form='(a15,3x,f9.5,3x,f9.5,3x,f6.2)' for i=0,numtarg-1 do printf,unit,format=form,$ targid[i],radeg[i],decdeg[i],vmag[i] close,unit
Note: you can check the output formatting by doing a test print to your terminal, using statements like:
form='(a15,3x,f9.5,3x,f9.5,3x,f6.2)' for i=0,numtarg-1 do print,format=form,$ targid[i],radeg[i],decdeg[i],vmag[i]
forprint is a handy AstUseLib utility for printing several vectors to either the screen or an ASCII file; the format keyword need not be specified.
Binary Files: See description of binary files above. The basic IDL routine for writing unformatted binary files is writeu . To create files that are portable between computer operating systems, use the optional xdr keyword with the open procedure.
FITS files: To write a FITS file from IDL variables, use fits_write .
If the images have been manipulated and you did not employ the AstUseLib header-tracking commands such as hextract, hrot, hastrom, hrebin, and so forth, then you must create a new header (use mkhdr ) or update the existing one to properly match the stored image. fits_write will optionally produce a simple header, or you can use the sx* commands, such as sxaddhist or sxaddpar to do this. It is particularly important to be sure that the parameters BITPIX, NAXIS* , and DATATYPE are properly entered. fits_write writes to the current directory.
SDAS or IRAF files: Use stwrt and irafwrt , respectively. These (deprecated) routines each produce two output files (*.hhh and *.hhd for SDAS; *.imh and *.pix for IRAF).
Other image formats
See the description above on how to use tvrd to copy the contents of a displayed image window to an IDL variable in preparation for writing to a file.
GIF format is a good way to store line graphics or processed images (in 2-D format) with special color tables embedded. JPEG, which stores images in 3-D format, is a natural means to save true-color images, though the compression algorithm used to reduce the file size may compromise quality (this can be adjusted). TIFF files store images in 3-D format in lossless form; they are used for publication-quality images. GIF and JPEG are the most widely used default formats for Internet browsers. Unfortunately, neither of these, nor the other popular commercial formats, provide a way to save header information.
Instant Storage of Session Data
You should delete all redundant or otherwise uninteresting variables (especially large arrays) before calling save . Review the save keywords before use. On a LINUX system, the output file will be in a universal XDR format which can be ported to non-LINUX computers.
save can be very useful for storage of intermediate results but is not recommended for permanent records. Why? For one thing, it's too easy to forget what all the variables mean; if you take the time to write FITS or ASCII files, you are more likely to document the work. For another, save encourages soaking up mass storage with redundant copies of arrays which may be only slightly changed from their original values. It is more efficient to create a script which makes relatively simple changes to images in preparation for further processing rather than to store the intermediate versions. Finally, the restoration of save files depends on the availability of IDL---not guaranteed if you move somewhere else. Standard IDL FITS or ASCII file-writing routines are preferable for permanent data.
The default name of the output file is idl.dat, and this file name will be overwritten on the next save . Best to change to something informative like ngc1068.sav.
These routines do not include the improved features of the DAOPHOT 1991 or later releases. To do PSF-fitting deconvolution of blended images, you can try the IDL package StarFinder or use the more recent stand-alone DAOPHOT or DOPHOT releases. The basic IDL routines remain very useful in preliminary assessment of data frames, prior to setting loose the mechanical scroungers, and in analyzing the results. (The output files of standard photometry programs can easily be read back into IDL.) IDL is also an excellent way to create synthetic data sets with known properties on which to verify photometry package operation.
Thanks to the efforts of Don Lindler, Wayne Landsman and others at GSFC, the IDL Astronomy User's Library offers convenient and powerful access to on-line databases. Information from these can be directly incorporated into your IDL sessions. For instance, the routine imdbase marks the locations of sources in selected catalogues on your current image display (assuming you have accurate astrometry for your image.)
In order to use the databases, you must have defined the environment variable $ZDBASE to point to the directory containing them.
To see what databases are available, use the command dbhelp,1 . To see what information is included in a given database and to display information for selected entries (here numbers 10,100,1000), type
dbopen,'[data base name] dbhelp,1 dbprint,[10,100,1000],'*
To retrieve and use the data entries, you will need to use the more sophisticated commands described in the documentation cited above.
The creation of new databases using the provided tools is relatively straightforward, so you can convert other datasets to online IDL databases as you wish.
IDL makes plots by default in the "current" terminal graphics window, which can be adjusted in size, location, etc, by using the same basic commands as for image displays (see above). Plots are scaled to the dimensions of the current window.
IDL Reference Guide
or on-line by typing ?plot and then selecting "Graphics Keywords."
xrange and yrange : 2-element vectors giving the minimum and maximum for each axis. If not defined, autoscaling will occur. Corresponding system variables are !x.range and !y.range .
/xlog and/or /ylog : set these keywords to use base-10 logarithmic plotting on the corresponding axis.
linestyle : selects style of line drawn (solid, dotted, dash-dot, etc) if psym=0 .
xstyle and ystyle : set axis options (e.g. exact range rather than rounded off)
!p.title , !x.title , and !y.title : strings for the plot title (centered over upper x-axis), x-axis title, y-axis title.
font or !p.font : specifies the font to be used for annotations. The intrinsic IDL-supplied fonts are "vector-drawn Hershey" fonts. These are satisfactory for screen displays, are device-independent, and are necessary when making 3-D plots. But it is better to use the "hardware" fonts supplied by graphics vendors for publishable quality work. Hardware fonts are associated with each of the various supported output devices. !p.font=-1 (default) selects the vector-drawn fonts; !p.font=0 selects the hardware fonts. To choose a particular font, once you have selected hardware-defined fonts, use the device,set_font="[name]" command.
The IDL defaults are not as "nice" as those in SUPERMONGO, for example. However, you can quickly customize to obtain as sophisticated a plotting style as you like. All the functionality of SUPERMONGO and other scientific graphics packages is inherent in IDL. Many of the 2-D and 3-D graphics routines are illustrated in the IDL Demos that come with the system.
Color tables for plots:
Note: it's easiest to use pseudo-color or emulated pseudo-color ( device,decomposed=0 ) for plots. In 24-bit true-color ( device,decomposed=1 , you must specify colors as longword integers.
Sample plotting scripts for displaying plots on your terminal follow. These involve a mixture of intrinsic IDL and AstUseLib routines:
x=findgen(10000)/10. ; Create the independent variable, ; here running between 0 and 1000 ; in intervals of 0.1 y=x*sin(x/15.) ; Create the function of interest plot,x,y ; Plot the function against the ; independent variable with a solid line. ; Plot will appear in the current window ; (or Window 0 if none are open) with axes ; scaled to match the maximum x,y ranges. plot,x,y,xrange=[0,50] ; Restrict plot to x values in range ; [0,50]. Y range automatically scaled. plot,x[99:199],y[99:199] ; Restrict plot to the 100th through ; 200th elements of the arrays. plot,x,y,xrange=[0,5],psym=1 ; Restrict plot to x values between ; 0 and 5, and plot points separately ; as plus signs. oplot,x,y ; Add continuous line plot to previous display, ; using existing axis scales plot,x,1+y^2,/ylog ; Plot a new function (defined to be positive ; definite), using a base-10 log scale on the ; y axis
Assume that the (ordered) vectors sb , rads , and sberr already exist, with sb and sberr in units of ergs/s/cm^2/Angstrom/arcsec^2 and rads in units of arcsec. Assume that because of bad data points you must truncate the plot to eliminate the first 3 entries and those after the 20th.
mags=-2.5*alog10(sb(3:19)) -21.1 ; Convert SB to magnitudes per arcsec^2 ; in the STMAG system, ignoring bad data. ; Assumes all SB entries are > 0. r25=rads(3:19)^0.25 ; Compute fourth root of radius vector, ; ignoring bad data entries magerr=1.086*sberr(3:19)/sb(3:19) ; Convert uncertainties ; in SB to magnitudes !y.range=[25,18] ; Set a non-default y-axis range; in ; this case the magnitude scale has ; smaller (brighter) values higher ; on the y-axis ; Make the titles !p.title='Sample Surface Brightness Profile" !x.title='Radius(arcsec)^0.25' !y.title='Surface Bright (mags/arcsec^2)' plotsym,4,1.5,/fill ; Choose filled triangles for plotting symbol, ; 50% larger than the default ploterror,r25,mags,magerr,psym=8 ; Make plot on terminal screen with error bars; ; Psym=8 specifies a user-created symbol, which ; in this case was defined by plotsym. ; No connecting lines between points. ; If psym were omitted or set to 0, no ; symbols would be plotted and the points ; would be connected by straight lines. xyouts,1.5,20,'NGC 4151' ; Add a label within the plot area. The ; positional parameters (data units) set the ; leftmost position of the displayed string. ; Choose these so the string is clear of ; the data and axes.
chan,3 ; Open plotting Window 3 !x.title='X' ; Make titles !y.title='Y' !p.title=' Contours for Image' square ; Set aspect ratio to make a square plot ; (Note that you must also give this command ; after the PostScript device is called ; when making hardcopies.) smooth_one=smooth(image,5) ; Smooth the image by a 5 pixel boxcar clev=[10,20,40,80,160] ; Define trial contour levels--assume ; these values span the range of interest contour,smooth_one,levels=clev ; Do fast test of contour plot. Check & ; iterate clev for best appearance contour,smooth_one,levels=clev,/follow ; Do more accurate (slow) plot, ; with labels
Assume x is a vector containing values in the range 0 to 5 and that y is the corresponding dependent variable. Assume that these are to be read from an ASCII file named xy.dat, which contains x and y in separate columns. xy.dat can contain an initial explanatory section and other separator headers, as long as none of these contain only one or two floating point numbers (since readcol will mistake those for data lines). The data columns need not be aligned. By default, the readcol routine will read in the numerical x,y data ignoring any line containing alphabetic characters. No labels are put on the plot in this example.
readcol,'xy.dat',xx,yy ; Read data from the file. Note that a format ; statement and 'open' command are not required, ; nor do xx and yy have to be pre-defined index=sort(xx) ; Sort the arrays in order of increasing x x=xx(index) y=yy(index) chan,1 ; Open Window 1 for plotting plot,x,y,psym=5 ; Plot the data with open triangles quadcoeff=poly_fit(x,y,2) ; Derive coefficients for best quadratic ; polynomial fit print,quadcoeff ; Print these out (optional) [Note: ; quadcoeff is a vector] tx=findgen(101)/20. ; Create independent variable vector for fitted ; values (uniform x interval of 0.05 units) quadtesty=poly(tx,quadcoeff) ; Create the fitted quadratic values oplot,tx,quadtesty,psym=1 ; Overplot the quadratic fit with plus signs cubcoeff=poly_fit(x,y,3) ; Derive coefficients for cubic fit cubtesty=poly(tx,cubcoeff) ; Create the fitted cubic values oplot,tx,cubtesty,psym=3 ; Overplot the cubic values with small dots ; [Assume the quadratic fit was adequate] delta=y-poly(x,quadcoeff) ; Compute the difference between y data and ; the best quadratic fit nix=where(abs(delta) gt 2.) ; Locate those y values which are more ; than 2 units from the best fit weight=fltarr(n_elements(x))+1. ; Create a weight vector corresponding to ; x with unit entries weight(nix)=0.0 ; Give deviant points zero weight newquadcoeff=polyfitw(x,y,weight,2) ; Derive improved quadratic coeffs. ; employing weights just assigned final=poly(tx,newquadcoeff) ; Create improved fit values chan,2 ; Open Window 2 for final, clean plot. ; (Window 1 is retained for comparison.) plot,x,y,psym=5,xrange=[1,3.5] ; Plot the data with open triangles in ; Window 2. ; Assume interest is limited to only ; part of data x-range. oplot,tx,final,psym=0 ; Overplot final fit with solid line
Suppose you have measurements of a quantity val for two samples of objects with a similar range of values and you want to compare the histograms of the two in a single plot. A simple way to do this in IDL is to create the union of the two datasets for a first plot and then overplot a shaded histogram of the second set. If the data consist of two vectors, val1 and val2 ,
allval=[val1,val2] plothist,allval,bin=0.2,xrange=[0,8] plothist,val2,bin=0.2,/over,/fill,fcolor=200
In the resulting plot, the val1 entries will be unshaded, while the val2 entries will be shaded with the fill color value 200 (the on-screen appearance will be determined by the pre-loaded color table).
Similarly, in the case of a subset of data values that are to be distinguished by some second parameter:
plothist,val1,bin=0.2,xrange=[0,8] good=where(param ge 0) plothist,val1(good),bin=0.2,/over,/fill,fcolor=200
The most common method of obtaining hardcopies or permanent storage of graphics output (plots or images) is to use PostScript files, since these can be printed on most laser printers. PostScript files can be later edited and reformatted, though special (non-IDL) programs are needed.
IDL also supports output of GIF, JPEG, TIFF, PNG, and other file formats. GIF and JPEG are standard for Internet Web browsers. GIF is recommended for line-drawing and plot output storage. TIFF is recommended for high quality, full color reproductions (e.g. transporting image files to vendors for hardcopies or publication).
You should always experiment on the terminal screen with your plot format before dumping to an output file. It is easy to do this by working out the set of commands you want while plotting to the screen, then typing set_plot,'ps (in the case of PostScript output) and repeating the commands using the command recall buffer.
For more complex plots, use the journal utility, then edit and re-execute the resulting file (or cut and paste across windows).
The set_plot command determines which output graphics device you are using. The most common versions of this command are:
set_plot,'x : Send output to X Windows (default) set_plot,'ps : Send output to the PostScript file "idl.ps"
The subsequent commands for sending data to the PS file are (mostly) the same as for putting data on your monitor screen, since monitors and PS files are interchangeable output devices for IDL.
You can always check the properties of the current graphics output device by typing help,/dev . You can change these defaults by using the device command. The hardware/software interfaces are sometimes non-trivial, and you will want to plan for a significant learning curve in doing things which are not "vanilla." Before sending large jobs to printers, vendors, etc., be sure to check the files using LINUX ghostview, xv, or other screen display programs.
Here are some graphics output methods for common situations:
set_plot,'ps ; Plot will be sent to PostScript file "idl.ps", ; not to the screen ; Note that your definitions of plotsym and ; system variables such as axis labels ; are still in force !p.font=0 device,/helvetica,font_size=12 ; Use Helvetica PostScript font for labels ploterror,r25,mags,magerr,psym=8 xyouts,2.5,17,'NGC 4151' ; Make PS plot with error bars, adding the ; label inside the plot area device,/close ; Close PS file $lp idl.ps ; Send file to printer $mv idl.ps surbriteplot.ps ; Rename PS file to save it. set_plot,'x ; Send further output to the terminal cleanplot ; Reset the plotting system variables to defaults
pic = bytscl(bigim,min=0,max=1835,top=255)--> picname = rotate(picname,3) .
PostScript files are the best way to get high quality line-drawing plots and are generally well treated by publishers. Grayscale or color images are another matter, however, and should be approached in an iterative way. A published PS image may look very different from the one you printed locally. GIF or TIFF files may yield better results than PS.
There can also be difficulties with the finite resolution of screen displays copied using the tvrd routine. This is true for the quality of both the image and any lettering which may have been added to it. On a typical computer screen, you will not get more than about 900x900 resolution. However, a PostScript file at 300 dpi can yield much higher resolution (1800x1800 in a 6" image).
If you store duplicates of your graphics output in GIF, TIFF, JPG, etc files, you can use non-IDL utilities like xv , Gimp , Photoshop , etc. to manipulate them further: in size, rotation, contrast, color table, and by applying various image enhancements. xv can convert GIF to PostScript and vice versa. Especially useful for compressing image sizes for use on the Internet, the arXiv Preprint Server, and so forth.
Not many people have experienced fully interactive computing before they start to use IDL. There are tremendous advantages but also many pitfalls for the unwary. The pitfalls will, of course, mostly seem obvious and trivial in retrospect---i.e. after you have learned to avoid them. A number of tips & warnings for IDL beginners are discussed in this section.
Paths, Procedures, Directories
Special IDL Interactive Aids & Accelerators
If, instead, you simply want a count of the number of entries with values less than 1, type:
Definition & Stability of Variables
a=fltarr(100,100)+10. creates a 100x100 array containing 10.0 everywhere.
Create vectors with simple definition statements like z=[1,3,9,27] ; statements like z=[z,81] enlarge vectors.
You can extract subarrays using index notation, as in parta=a[100:156,200:296] .
For instance, x=5/2 returns the value 2 (integer) while x=5.0/2 returns 2.5 (floating point). x has different values and attributes in consequence. Watch out! For insurance, a good habit is always to enter numbers in floating point calculations with decimal points. E.g. diam = 2.*radius .
The result of a computation involving mixed types is influenced by the order in which IDL evaluates an expression:
IDL>print,6.*500.*70.*70. 1.47000e+07 IDL>print,6*500*70*70 19936 IDL>print,6*500*70*70. 937440. IDL>print,6*500*70.*70 1.47000e+07
A=fltarr(200,200) ; Define floating point array with all elements ; = 0.0 A(*,*)=1. OR A=A+1. ; Proper way to fill array A with 1.0 A=1.0 ; Wrong way to fill array A with 1.0. ; You have just changed A to the scalar 1.0!
To eliminate the orphan dimension here and obtain a standard row vector, use the reform function: b=reform(b) . You could also use the transpose function to convert the column vector into a row vector.
You can check the current structure for a variable using the help command.
print command prints row vectors across the screen and column vectors down the screen.
find=where(time eq 1000.) norm0=value(find) normalval=value/norm0
Naively, you expect norm0 to be a scalar, as it would be if you typed norm0=value(151) , for instance. However, if you try this, you will find that here it is not a scalar, but instead has type array(1). This is because the where routine returns a vector, which definition propagates through to the definition of norm0 . The resulting quantity, normalval , which divides two vectors of different lengths, is therefore a scalar, not a vector! The reform function is no help here. To get the intended result, you must define the normalizing constant as: norm0=value(find[0]) .
norm0 to a scalar using an array function with a scalar output: e.g. norm0=total(value(find)) or norm0= max(value(find)) . -->
If you are writing code for general application, you would also want to confirm that find returns only a single non-negative element. The number of elements returned by the where function can be determined, for instance, as follows:
find=where(time eq 1000.,count) if (count eq 1 ) then begin.
Differences from FORTRAN & C
Other Issues
Programming Philosophy
Before executing IDL, you must define a special set of IDL environment variables, so that LINUX and IDL know where to look for the various IDL executable and program packages. Normally, you do this by source -ing a special idl_env file. Default environment files are contained in each IDL distribution from Exelis.
To use the UVa Astronomy Department system defaults, give the command: source /astro/idl/idl_env.csh . (For a Bash session, use idl_env.sh .) You will be automatically set up to use the latest version of IDL including the MOUSSE package as installed on our local servers. If you are happy with the system defaults, you can ignore the rest of this section, except for morbid curiosity.
On the other hand, if you want to customize your IDL configuration, you need to know the details of the various environment variables.
Instead of using idl_env, you could define the various required IDL environment variables in your .login or .cshrc files. However, it is cleaner to keep all such definitions together in a separate idl_env file.
As an example environment setup file, the standard UVa file is shown under Appendix B below. /astro/idl/idl_env.csh . (For a BASH session, use idl_env.sh .)
You must also specify the location of the IDL license file that authorizes your host to execute IDL. Licenses are issued for single or multiple hosts. The UVA license supports 50 hosts simultaneously. The environment variable pointing to the license file is named $LM_LICENSE_FILE .
You must also specify the search path for all *.pro routines you wish to use (other than the intrinsic Exelis routines).
Once $IDL_PATH has been defined, it cannot be changed during an IDL session.
IDL's default in the absence of a specified path is to use the current directory only.
When IDL looks for a named routine, the path is searched in order from the first entry to the last. The first file with the expected name is executed.
The search path should include the IDL User's Library (distributed with native IDL), the IDL Astronomical User's Library, the ATV directory, and the MOUSSE directory.
The path should also include your own idl directory and any other directories containing customized routines which you wish to invoke without making them the current directory.
Beware of multiple versions of the same program (e.g. an AstUseLib version and a Users Library version)! You must arrange your path so that the preferred version occurs first in the path.
At UVa, the AstUseLib versions are preferred, and should be placed first in your path except for your own idl directory.
Recommended path structure (see examples below for syntax):
[current directory], [your idl directory], [other special local *.pro libraries, if any], [Astronomy Users Library], [MOUSSE], [ATV], [IDL Users Library].
Separately, if you wish to use the databasing software, you must specify the location of the IDL databases (the zdbase directory)..
In order to customize your IDL session further, you can define the $IDL_STARTUP environment variable. The file to which this variable points will be executed at the beginning of your IDL session. This startup file can execute an arbitrary number of other IDL programs and scripts. The default at UVa is to execute the startup file mousse_startup.pro . Among other items, this executes the procedure cinit.pro ; many MOUSSE routines will fail if cinit is not executed.
Finally, the idl_env file also establishes aliases for the various IDL executable files.
mousse_startup.pro ), or you can specify a personalized version which includes all the statements in the default plus other customizing statements. The startup file executes the MOUSSE procedure cinit.pro ; many MOUSSE routines will fail if cinit is not executed.
### This is the UVa Astronomy LINUX system default env file ### as installed on the department server, March 2009. ### Location is: /astro/idl/idl_env.csh # C shell commands to define IDL environment variables and aliases. # # Script name: idl_env.csh # # This script is used by C shell users to define # the environment variables and aliases required by IDL # related commands. # # To execute this script, give the following command; or, # for automatic execution, place it in your .cshrc file. # # source /astro/idl/idl_env.csh # # To overwrite the defaults set in this file, you can source other # setup files from your individual .cshrc file. # ##### # System choice - deprecated 03-07-11 ##### # set sys=`uname -a | awk ''` # if ( $sys == 5.6 ) then # setenv IDL_DIR /net/astsun.astrosw/idl/ # else # setenv IDL_DIR /net/jeeves.common/rsi/idl/ # endif # Note: the March 2009 configuration substitutes a departmental version # of IDL v7.0 (/astro/itt/idl) for the UVa v6.4 version # (/common/rsi/idl). #setenv IDL_DIR /common/rsi/idl/ setenv IDL_DIR /astro/itt/idl/ setenv IDL_HOME $ #setenv IDL_HELP /common/rsi/idl/help setenv IDL_HELP /astro/itt/idl/help # License file #setenv LM_LICENSE_FILE /common/rsi/license/license.dat setenv LM_LICENSE_FILE /astro/itt/license/license.dat # Local packages setenv ASTROLIB_DIR /astro/idl/Astrolib setenv MOUSSE_DIR /astro/idl/Mousse setenv MOUSSE98_DIR /astro/idl/Mousse.98 setenv PIA_LOC_GEN /astro/idl/PIA/ setenv ATV_DIR /astro/idl/atv setenv FUSE_DIR /astro/idl/FUSE setenv IDL_HELP_PATH "+$:+$:+$" # ZDBASE is the location of the IDL-formatted databases setenv ZDBASE /astro/idl/zdbase # Notes on IDL_PATH # # Symbol "+" means that the path will be expanded to include all # subdirectories within the given directory. # The user's own IDL programs are assumed to be in a directory named, # or linked to, "idl" under the user's home directory # Directory $IDL_HOME/lib contains the IDL User's Library of # standard user-written utility routines. setenv IDL_PATH ".:+~/idl:/astro/idl/UVAlocal:+$:$:+$:+$:+$/lib:+$/examples:+$:+$" setenv UIT_DATA $/data setenv IDLUSR $ setenv IDLUSER $ alias idl $IDL_DIR/bin/idl alias idlde $IDL_DIR/bin/idlde alias idldeclient $IDL_DIR/bin/idldeclient alias idlhelp $IDL_DIR/bin/idlhelp alias idlrpc $IDL_DIR/bin/idlrpc alias insight $IDL_DIR/bin/insight alias idldemo $IDL_DIR/bin/idldemo alias pia $PIA_LOC_GEN/PIA # The following IDL startup file will be executed before # each IDL session. User can specify a different # startup in his/her .cshrc file. setenv IDL_STARTUP /astro/idl/startup/mousse_startup.pro
"Startup" files are batch files which are automatically executed when IDL begins running. They enable to user to customize his/her IDL session.
The startup file is often placed in $IDL_DIR . However, it can be anywhere. In order for IDL to locate it, you must point the environment variable $IDL_STARTUP to it. If this variable is not defined, no startup file will be executed.
This particular example defines special "systems variables" and common blocks used by the MOUSSE image processing package. These improve the efficiency of the package and reduce the number of typed entries the user must make to use it.
Contents of mousse_startup.pro:
; STARTUP FILE mousse_startup.pro ; Last Modified: 15 May 1992, 30 Sep 93, 27 Dec 93, 22 Aug 97 print , "Running mousse_startup.pro. " print , "For help on AstUseLib and Mousse routines, use the Mousse MAN procedure." setplot, 'X' DEFSYSV,'!DEBUG',0 DEFSYSV,'!TEXTUNIT',0 DEFSYSV,'!TEXTOUT',1 DEFSYSV,'!PRIV',0 defsysv,'!psprinter','astro-hp' ; UVA default printer cinit ; CINIT sets up MOUSSE common blocks !PROMPT = 'IDL>' !EDIT_INPUT = 100 on_error,1 ; Return to main program in case of error print,' *** on_error,1 is default ***' print,' *** If terminal is vt100/tek, type SETPLOT,0'
This guide assumes you are executing IDL from within an X-WINDOWS environment. So: first, you must start your windows system and open an appropriate window to enter IDL commands. The location of display windows opened by various IDL applications can be controlled in many instances. Consult the description of xpos under the window command. It is useful to place display windows away from your command window, wherever possible. You can move existing windows using your mouse.
On an Apple MAC, you must ordinarily manually start the X11 program. You can select a monitor display class from its Preferences menu; changes here will not take effect until after you exit and re-start X11 . You must execute IDL from within an X-window in order to use graphics.
setenv DISPLAY [local]:0.0
where [local] is the fully qualified network name of your local machine, e.g.
setenv DISPLAY bonkers.astro.virginia.edu:0.0
This Guide is copyright © 1991-2015 by Robert W. O'Connell. All rights reserved.
Reproduction or distribution without attribution and commercial uses are prohibited.
Contents last modified June 2012 by RWO. Email: rwo [at] virginia [dot] edu