TOPCAT Full Changelog

What's new in TOPCAT Full 4.9

Nov 1, 2023

New functionality:
New Corner Plot Window added.
The authentication handling has changed, and it is now possible to log in to TAP services that comply with the (draft) "SSO_next" proposal for advertising authentication methods in the VO. The star.basicauth.user/star.basicauth.password system properties can no longer be used to set authentication information globally for the application (this was a rather insecure practice anyway).
Column metadata export actions (Import as Table, Save as Table) added to Columns Window.
You can now Search for columns by metadata in the the Columns Window. This is available as the Search Column action in the toolbar and Columns menu, and also using the new popup menu available from the metadata display JTable.
Minor enhancements and behaviour changes:
MacOS topcat-full.dmg file should now work for both Intel and Apple Silicon architectures.
Add new Sky + X with Errors Match Algorithm option.
Update hard-coded Registry URLs for Euro-VO and STSci to use https not http protocol.
Avoid annoying popup in Cone/SIA/SSA windows when position is resolved before radius is filled in.
Flag ConeSearch 1.03-style error result tables.
Column names with embedded carriage returns are now recognised in column selectors.
Bugfixes:
Fix a bug in the Pair Match and Multiple Match windows which assigned the created matched/match<N> subsets to completely the wrong rows. These subsets are only created for some match types, and everything else about the matches was unaffected. STILTS was not affected. This was a regression bug since v4.8-4. Thanks to Claire Greenwell (Durham) for reporting this.
Fix bug which ignored some points when painting multi-threaded, multi-dataset or multi-subset plots. This bug was introduced in v4.7 and was supposed to be fixed at v4.8-8 but wasn't.
Fix TAP window to select ADQL-2.0 not ADQL-2.1 if service declares no ADQL versions.
Fix TAP capabilities parsing to identify language features with non-standard capitalisations.
Fix fencepost error in reverse cumulative histogram plotting.
Fix missing secondary X axis bug in stacked Time plots.
Fix bug in histogram plotting that could cause crashes for small ranges far from the origin.
Fix aux axis/legend positioning issue in sky plot (regression bug introduced at v4.8-8).
Fix bug in ADQL syntax highlighting related to delimited identifiers (quoted table names). This bug was only introduced with the VOLLT upgrade in the last release (TOPCAT v4.8-8).
Fix issues to do with sky coordinate system choice that sometimes caused SkyGrid and Healpix layers to generate broken STILTS commands for export.

New in TOPCAT Full 4.8 (Jan 12, 2021)

In this version, the table handling library STIL has been upgraded to v4.0, which enables some enhancements including multithreading and new I/O handler features.
Notable new functionality:
Auto file format detection now examines filenames to help guess format; this means that e.g. when loading CSV files named with the ".csv" extension, it is no longer necessary to select "CSV" in the Load Window Format selector.
Scheme specifiers can now be entered in the Load Window to load tables not corresponding to external files. Options currently provided include simple sequence, simulated sky and strange attractor data.
Add more example tables (using skysim and attractor schemes) to the Example Tables menu.
Table input/output handlers specified in the table Save/Load Windows can now in some cases take parenthesised options, e.g. you can type in "votable(version=V12,format=BINARY)" instead of just selecting "votable". For examples see individual handler documentation, e.g. the VOTable output handler and ECSV input handler.
Improved performance when multiple threads are accessing synthetic columns or subsets concurrently.
Statistics calculations in Statistics Window now execute in parallel.
New Approximate Quantile Calculation option in Statistics Window that runs in limited memory.
Minor enhancements:
COOSYS and TIMESYS attributes are now preserved during VOTable I/O for table PARAMs (as well as for FIELDs, which was already the case). However, they are not currently displayed in the application GUI for table parameters, only for columns.
The Table I/O section has been somewhat reorganised and tidied up.
Add new options acos and cos for Scaling in the Aux Axis Control etc; these provide linear-like stretch functions with steeper/flatter ends, which may be useful for shading by quantities with most variation near to/far from the middle of the range.
Perform part of the Sort operation in parallel.
Slightly improved documentation for CDS Upload X-Match Window.
Add Plot|Parallel Caching menu option in plot windows; this is experimental and not generally recommended.
Bugfixes:
Upgrade JEL to v2.1.2: fixes a bug that could cause trouble when reloading saved sessions with certain boolean-valued synthetic columns: load could fail or values could be inverted. Also some efficiency improvements in object creation.
Fix histogram ranging bug (plot failure under certain circumstances).
Avoid inserting NULL_VALUE custom metadata entries into ECSV output.
Fix annoying column selector bug that sometimes required an extra click.
Fix up MacOS application packaging issue so that (hopefully) topcat/stilts startup scripts work on Mac M1 architecture.
Cope better with infinite values in aux plot coordinates.

New in TOPCAT Full 4.7 (Nov 19, 2019)

Runtime environment: Java 8:
From this release, TOPCAT requires Java 8 (a.k.a. Java 1.8) or greater to run, rather than Java 6 as for previous releases. Java 8 has been around since 2014, so it should be available on all but very ancient platforms. If startup fails with a java.lang.UnsupportedClassVersionError then you need to upgrade.
New or enhanced funcationality:
Most of the plotting, though not the intial data preparation, will now run in parallel for large (>1e5 row) datasets. This should make interacting (e.g. pan, zoom, change settings) with slow plots faster on multi-core machines. (In rare cases this multi-threading might cause problems with memory usage; it can be effectively turned off if required by using the system property java.util.concurrent.ForkJoinPool.common.parallelism.)
New activation actions Display HiPS Cutout and Send HiPS Cutout; these make it easy to access imagery for many surveys using the excellent hips2fits service from CDS.
New class URLs contains expression language utility functions for constructing certain service URLs: hips2fitsUrl, bibcodeUrl, doiUrl, arxivUrl, simbadUrl and nedUrl. Existing functions urlEncode and urlDecode have been moved to URLs from class Strings.
Rework the View in Web Browser activation action so that it can show web pages for not only URLs and filenames, but things like DOIs, bibcodes and arXiv identifiers too.
New JavaFX browser option in the View in Web Browser activation action. This is like the existing Basic Browser, but more capable, e.g. it can render JavaScript. Note JavaFX is not available with all Java installations.
Add Pause Sequence action to Activation Window.
Activation Window sequence actions now update activated row (display highlight in table viewer and plots) as they progress.
New Delay activation action added as a convenience for use with "slideshows".
Add tiny Show/Hide All buttons at the bottom of the subset list in the Data Layer Control Subsets tab.
Add new Use Position configuration option to polygon plot form to toggle inclusion of reference position in polygon vertex list.
Add new Fix Errors button for common ADQL errors in the TAP Window.
Modified behaviour for offset (e.g. unsigned) longs in FITS files. 64-bit integer columns (TFORMn='K') with non-zero integer offsets (TSCALn=1, TZEROn<>0) are now represented internally as Strings; previously they were represented as Long integers, but values out of the possible range appeared as null (with a warning written through the logging system). Such columns are most commonly seen representing unsigned long values. If written back out to FITS, the offset long value will be reinstated, but other output formats cannot encode unsigned longs, so they will stay as strings.
New functions parseBigInteger and parseBigDecimal in class Conversions.
Minor enhancements and behaviour changes:
VOTable output now writes VOTable version 1.4 by default (was 1.3).
Modified plot legend display so that small markers are displayed a bit bigger in the legend than on the plot for readability.
Add new colour map Cividis.
Upgraded ADQL parser from 1.4 to VOLLT 1.5-1. Should result in minor improvements to TAP ADQL syntax highlighting.
Permit FITS and VOTable files with zero-length string columns. Previously all-null or zero-length string columns were sometimes forced to single-character values.
Fix one RegTAP example query in accordance with RegTAP-1.0 Erratum #1.
Add more detail (response content) to certain bad response TAP error messages.
Update mapped file unmapping implementation to work for java9+.
Minor changes to behaviour when querying TAP 1.1 services (REQUEST parameter is omitted).
Deprecate Display Cutout Image activation action; Display HiPS Cutout is usually a better choice.
The front page screenshot montage (which was very out of date) is replaced by one with up-to-date screenshots.
Registry queries for standard services now search for intf_role='std' instead of intf_type='vs:paramhttp', which is the recommended pattern for RegTAP 1.1, and believed to work for current contents of known searchable registries.
Eliminate unnecessary "(no position)" text in Display Image activation action log message.
JSAMP to version 1.3.7.
Bugfixes:
Avoid sometimes losing precision when reading ASCII/CSV values in the range +/-(1e-45..1e-38).
Fix TAP window to avoid locking up GUI in case of slow loading service logo image.
Fix Execute Code activation action so that the Synchronous option is set false by default (this was a bug - true can cause responsiveness problems for slow commands).
JEL bug fix update, to avoid unwanted debugging output for String function null returns.
Fix Quantile plot form bug (no non-JIDE RangeSlider fallback) that stopped it working in some environments such as Java 10 on OSX and Nimbus L&F.
Fix load error encountered when specifying a Cone Search service by URL rather than selecting from a registry query list. This was a regression error since v4.6-2.
Fix GUI updates when current subset expression is edited: data window view and apparent row count are now updated accordingly.
Fix plot axis ranging bug: padding was not always applied properly for logarithmic axes.

New in TOPCAT Full 4.6 (Apr 26, 2018)

Activation and DataLink:
There has been a complete overhaul of the Activation Action system, partly motivated by the need to work with the DataLink standard, and some related DataLink functionality introduced:
A new Activation Window replaces the old one, with more flexibility, better configurability, better reporting, and better extensibility.
You can now configure multiple activation actions at once.
The result (success/failure plus detail message) is visible for each action performed.
New general-purpose actions available: Load Table, Run System Command, Send VOTable.
New DataLink-related actions: Invoke Service, Invoke Datalink Row, View Datalink Table.
The new framework is also much more extensible, so other actions will be added in the future, and it's also possible to plug in your own at runtime.
A new Datalink Window is added for viewing {links}-response tables and invoking their links.
DataLink-style Service Descriptor RESOURCEs in input VOTables (and FITS-plus tables) now become table parameters; they can be viewed in the Parameter Window with class ServiceDescriptor, and will survive a VOTable Save/Load cycle.
Note at this release the Activation Window configuration is not persisted, so you have to set it up every time you load a table. In future releases, it is planned to save configuration state with table sessions.
New expression language functions:
Add class Gaia to the expression language. This contains functions for estimating distances from parallaxes and propagating astrometric parameters and errors to different epochs. These astrometric functions are not specific to data from the Gaia astrometry satellite, but are presented in a form convenient for use with the Gaia DR2 source catalogue.
New array manipulation functions slice and pick added to class Arrays.
New utility function square added to class Maths.
Plotting enhancements:
Various improvements to the Contour plotting form. You can now contour quantities weighted by a given coordinate, rather than just point density. The smoothing from weighted point samples to the grid being contoured can be performed using sum, mean and other combination methods, it now uses a Gaussian kernel rather than a square top hat, and performance is considerably improved especially at large smoothing widths. The contour levels used are now reported, so can be seen at the bottom of the Form tab in the control panel. The contours are now plotted correctly right up to the edge of the visible plot, and various bugs have been fixed.
The Histogram plot now offers the Combine option (sum, mean, median, min, max, stdev, etc) for weighted histograms. This means that you can plot e.g. the mean value of a given quantity per interval on the X axis rather than just the summed quantity. A corresponding (though somewhat less well-defined) option is also provided when plotting KDEs and Densograms.
The Combine option that configures how values are binned in various histogram-like plots (SkyDensity, Healpix, Grid, and since this version also Contour, Histogram, KDE) now has two new options, sum-per-unit and count-per-unit. These work like the existing sum and count options, but scale the combined values by the relevant unit (e.g. X axis unit or solid angle). Where these units are physical, a Per Unit parameter is also provided for scaling in convenient units: second, day, year etc for time (Time plot), and square degree, arcminute, arcecond etc for solid angle (SkyDensity, Healpix). In some cases, the default Combine values have changed.
The Histogram plot form now lets you export the calculated binned data per plot layer for histograms plotted in the Plane and Time plot windows as well as in the Histogram window (from the Report panel or Export|Layer Data menu).
The Linear Fit form is now available in the Time Plot window as well as the Plane Plot.
Minor behaviour changes:
The width of table selector widgets is now limited, so tables with very long labels no longer result in very wide windows.
Improvements in documentation of the expression language functions: in the Function documentation section classes are now listed in alphabetical order, and the documentation of each function is more complete, including descriptions of all parameters and return values, with examples in some cases. Some readability improvements have also been made in the Available Functions Window.
The Available Functions window icon is now a bit less ugly.
The IPAC table reader now matches data type specifications case-insensitively.
Adjust subset name assignment heuristics in Column Classification Window.
Updated URLs in Messier demo table.
Rearrange TAP|Metadata Acquisition menu options in the TAP window to reflect current practice and standards. The default behaviour (Auto) now uses VOSI-1.1-style scalable tables endpoint rather than VOSI-1.0 for relatively small services (this is still VOSI-1.0 compatible), the CADC option is withdrawn, and some of the other options are re-ordered.
Bugfixes:
Update JCDF to v1.2-3; fixes some CDF reading bugs.
Stop the right hand side of the subsets stack being obscured by the scrollbar when there are too many subsets to see at once.
Fix a bug in the Data Window Subset From Unselected Rows action - it was including all unselected rows rather than just the visible ones, as per documentation. This was a regression bug introduced at v4.2-1.
Prevent MultiCone window from sending cone search queries with sr=NaN.
Fix bug in VOSI 1.1 TAP metadata acquisition detail request URLs (spuriously doubled '/' character).

New in TOPCAT Full 4.5-1 (Nov 8, 2017)

New in TOPCAT Full 4.5 (Oct 2, 2017)

New functionality:
New STILTS Control in plot windows. This displays a STILTS plotting command that can regenerate the currently visible plot.
Saving and loading sessions now preserves algebraic expressions used to define columns and row subsets (previously only the fixed values were saved and the expressions were lost and could not be edited on reload). Note that sessions saved by this version cannot be loaded by older versions (though the other way round is OK).
New SkyGrid Layer Control can draw multiple sky system coordinate axis grids on sky plot.
New plot forms XYCorr and SkyCorr for error ellipses rotated as specified by Gaia-style correlation values.
Add click-header-to-sort action for a number of displayed JTables: the Columns, Statistics, Parameters and Subsets windows. This makes it easier to deal with very wide tables.
Add position angle calculation functions posAngDegrees and posAngRadians to expression language.
Table I/O changes:
It is now possible to write and re-read tables with >999 columns to FITS or colfits format. The new limit is 2^31 columns. This uses a non-standard convention; software that is not aware of the convention (e.g. CFITSIO or earlier STILTS versions) will only be able to use the first 998 columns of tables written in this way.
For VOTable columns that reference COOSYS elements, the relevant information is now visible (CoosysSystem, CoosysEpoch, CoosysEquinox) in the Columns Window.
Any columns referencing COOSYS elements read from VOTable-based formats (VOTable or FITS-plus) will now be written out to VOTable-based formats with equivalent COOSYS references included. Currently not table parameters though.
The default version for output VOTables is now VOTable 1.3. New output formats votable-binary2-inline and votable-binary2-href are now offered alongside the five previously available VOTable variants.
Slight changes to the FITS-plus output handler VOTable formatting in the primary HDU; now uses default output VOTable version rather than VOTable 1.1.
FITS keywords using the ESO HIERARCH convention can now be read as table parameters rather than ignored when reading FITS tables.
Minor enhancements and changes:
Added Durham Galaxy Formation Catalogue URLs to the Virgo-Millennium Simulation Query window (which has been slightly renamed from the GAVO Millennium Run Query window).
The Euro-VO RegTAP Registry is now included in the hard-coded list of options in VO window registry selectors.
Updated experimental TAP|Service Discovery|Reg Prototype menu option to match v1.1 of Data Discovery Note.
Replace option autoscale with unit in SkyVector and SkyEllipse sky plot forms.
Change autoscaling defaults for plot forms Vector, SkyVector, XYEllipse, SkyEllipse; autoscaling is now turned off by default. Apologies for this change in behaviour, but it's better for consistency with new layers XYCorr and SkyCorr, and presents less danger of misinterpreted plots.
Bug fixes:
Update PixTools (HEALPix) library to 2017-09-06 version (https://github.com/kuropat/eag-HEALPix, 447a7be073876dba32). This fixes a bug in healpixRingIndex that could give the wrong value for small values of longitude near zero. It seems possible that this might have led to very infrequent missed associations when crossmatching in these regions, but tests appear to indicate that no such errors would actually have resulted.
Fix misfeature in SkyVector and SkyEllipse plot forms; these now preserve orientation on the sky even when the View and Data Sky Systems differ. Previously the Delta Longitude/Latitude and Position Angle parameters were always interpreted in the View, rather than the Data, sky system.
TAP examples are now picked up from the /examples document using more lenient rules (no requirements for a containing vocab attribute).
TAP Examples display line now copes better with long and multi-line names from service-provided examples.
Long fields (>10240 characters) in output CSV files are no longer truncated.
If a table is received from SAMP with a table-id that has already been seen, it will not be loaded (a SAMP error response is returned).
Fix bug in plot title placement.
Fix bug when saving tables generated from plots (e.g. SkyDensity HEALPix maps) that sometimes saved data from the wrong plot.
Fix bug in Statistics Window table export/import actions; it was failing if the Sum statistic was included in the presence of boolean columns.

New in TOPCAT Full 4.4 (Mar 27, 2017)

Visualization improvements:
The GUI has changed in the Form tab of plot window Data Layer Control panels. To add (and remove) forms there is now a Forms button that pops up a menu rather than the toolbar with form icons that used to be there. This change was necessitated because there were getting to be too many icons to fit in the default width of the toolbar. But it's probably an improvement anyway since you can now see the name of each available form as well as its icon.
Improved the documentation of plot Forms and Shading Modes in the user document - each option now has a screenshot showing it in action.
The colour selector widget now comes with a new Free colour chooser button, giving you various ways to select any RGB colour you like.
New Grid plot form, to plot a 2-d weighted histogram. This replaces the Density form, which has been withdrawn (Grid can do all the same things and more, except specify bin size in screen pixels).
New Quantile plot form, which can (e.g.) plot median lines through noisy data.
New Gaussian plot form for Gaussian fits to histograms.
New normalisation (scaling) option unit for histogram, KDE, and KNN layers.
New normalisation (scaling) options per_day etc for histogram, KDE and KNN layers in the Time plot window only.
The Histogram control is now available from the Time as well as the Plane plot window, so you can now plot histograms with a temporal horizontal axis.
Various tweaks to the details of how plots are positioned on the screen or in output graphics files.
Miscellaneous enhancements:
Fixed the match score (distance measure) for the Sky+X and Sky+XY matching algorithms. Previously, the score was a linear sum of the unscaled distances for the constituent matchers, which meant a Best match was pretty meaningless. Now, it adds scaled distances in quadrature, so Best matching should give you a somewhat sensible result.
The Sky with Errors and N-dimensional Cartesian with Errors matching algorithms now report output separations as scaled (dimensionless) values rather than in physical units. This means the results are more comparable, so Best match results will make more sense.
The Maths function hypot now takes an arbitrary number of arguments (instead of exactly two).
Added STScI service to default list of RegTAP registry endpoints.
Minor improvements to the logic in the Concatenation Window that tries to match up columns between the two tables.
Update JCDF library to v1.2-2 (2017-01-01 leap second).
Bug fixes:
Fix KDE/KNN plotting bug that could get scaling badly wrong for normalised cumulative plots.
Fix nasty Concatenation Window bug; the current subset and sort order of the Appended table were not properly taken account of in the output. This was a regression bug introduced at v4.3-4.
Fix a regression bug (since v4.3-2) in the Frame layer control; values entered were being ignored for image export or display or both.
Fixed a bug that could show Error information for the wrong job in the TAP Running Jobs tab.
Fix TAP window bug that prevented job status from being updated in Running Jobs tab.
Fix small bug in Plane/Time plot windows; horizontal axis label was sometimes off the bottom of the plot by a few pixels.
Fix subpixel offset of colour ramp frame in PDF/PostScript graphics output.
Remove spurious padding from EPS graphics output.

New in TOPCAT Full 4.3-5 (Oct 12, 2016)

New in TOPCAT Full 4.3-4 (Oct 12, 2016)

Visualisation and HEALPix:
Add options (from layer control Report panel or plot window Export menu) to export the HEALPix density map calculated by a SkyDensity plot; either save it to disk or import it as a new table.
New Healpix layer control in the sky plot lets you plot tables that represent healpix maps (e.g. as exported from SkyDensity plot).
Add some HEALPix-related functions to the Tilings class: conversions from pixel index to sky position and conversions between ring and nested schemes.
Colour ramp Clip control (e.g. for the Aux Axis) now has a new Default checkbox. When checked, a default clip is applied, which avoids very light colours, and when unchecked you can use the double-slider over the whole range. Previously, the double-slider range was pre-clipped, so very light colours were not available.
Improve performance of some plot types. In weighted shading mode you can now adjust the colour map without appreciable delay. SkyDensity plots are somewhat improved.
TAP/VO enhancements:
The TAP window now honours the (TAP 1.1) column_index column if it is present in the TAP_SCHEMA.columns table. This means that columns can be displayed in a predefined, rather than random, order in the column metadata display tab if the service supports that.
The TAP window also now acquires and displays non-standard, as well as standard, table and column metadata items if provided by the TAP service. These come from non-standard columns in the TAP_SCHEMA.tables and .columns tables, or from attributes in custom namespaces in the TableSet document, depending on metadata acquisition policy. Services can use this if they wish to report things like table row counts or column data limits.
Add VOSI 1.1-style detail-sensitive /tables endpoint query options to the TAP|Metadata Acquisition menu in the TAP window.
Bug fixes and minor enhancements:
The GBIN input handler can now pick up more metadata from the classpath. For suitable tables, metadata included in datamodel classes if present can be interrogated to provide table and column descriptions and UCDs. There are still some deficiencies of this functionality (no column order, utypes and units missing, large file "temp.xml" written to current directory) dependent on issues in the upstream Gaia libraries and ICD.
Fix bug that caused read failures for large (>0.5Gb) FITS files outside of the current directory on 32-bit JVMs. This was a regression bug since v4.2-2.
Fix long-standing bug that failed to release file descriptors when opening FITS tables (could cause an error if very many FITS files were opened).
Improve the Concatenation Window. You can now use algebraic expressions for the appended table columns, and changes to the appended table no longer reset the window state.
Fix minor border-painting bug in sky plot window (sky outline painted on GUI components).
Combiner option stdev replaces variance in density, sky density and weighted plot layers.

New in TOPCAT Full 4.3-2 (Nov 27, 2015)

Crossmatching bug fix:
Fix a long-standing crossmatch bug relating to range restriction during pre-processing. This could have caused missed associations (but not false positives) near the edge of coverage regions when using per-row errors, if the scale of the errors differed (especially differed significantly) between the matched tables. It affected match algorithms Sky with Errors, -d Cartesian with Errors, 2d_ellipse and Sky Ellipses only. Thanks to Grant Kennedy (IoA) for reporting this bug.
Density plots:
Some more options for making weighted density plots have been added. Since v4.0 the Density shading mode has let you see the density of plotted points, but this lacked some features. Three new ways to do density plots are added; these all give you the option of weighting by an additional coordinate (like the Aux mode), choosing the combination method (mean, median, sum, max, ...), and displaying the quantitative value-colour mapping on the shared colour ramp (previously aux axis) beside the plot. The new density plots are:
Weighted shading mode, using shaped marker kernels on the screen pixel grid, available for all plot types
SkyDensity form, using HEALPix bins on the celestial sphere, for the Sky plot
Density form, using square N*N screen pixel bins, for the Plane plot
The details are somewhat experimental and may undergo some adjustments in future releases (feedback welcome).
Colour maps
There are various changes affecting selection and display of colour maps used for density and aux axis shading:
The default colour map for Aux mode, and other layers using the shared colour map, is no longer Rainbow! It's Inferno. Rainbow colour maps are much hated by visualisation experts. Of course you can still choose Rainbow if you like.
Add some new colour maps: Viridis, Inferno, Magma and Plasma from Matplotlib 1.5, the SRON rainbow variant developed by Paul Tol, some diverging maps (HotCold, RdBu, PiYG, BrBG) and a qualitative constant chroma/luminance map HueCL.
The options for Density and Aux shading are now mostly the same as each other except where there's good reason to differ. Previously they were different in haphazard ways. The order of the options is now a bit more rational too (similar maps are grouped together, more common ones near the start).
An attempt is made to give the default form of each colour map a sensible name, without leading minus signs.
Fix it so that the whole range of each map is distinguishable from white. This is a good idea when you're plotting symbols on a white background, which is common in topcat. Perhaps there are cases it's not such a good idea; if you think so, complain and I may change it back.
Try to fix it so that all the colour maps go in the same direction (light->dark) where applicable.
Throw out a couple of particularly useless colour maps.
Colour map ramp display is now different for non-absolute maps; their effect is shown on a selection of base colours, not just for one base colour.
Minor items:
Try harder to identify epoch columns (suitable for time plot), in particular look for VOTable xtype of JD or MJD, and units of year.
Add some functions to the Tilings class to do with solid angles (healpixSqdeg, healpixSteradians, steradiansToSqdeg, sqdegToSteradians, SQDEG).
Tweaked topcat startup script to work better if it's invoked from a symbolic link.
Fix the description in SUN/253 document of Sky ellipse and vector plot forms, previously they were not distinguished from the Plane variants of those forms.
Fix plot bug; titles were painted in white for pixel export formats.
Rationalise plot report logging. Some more diagnostic information about plots is now logged at the INFO level (visible if topcat is run with the -verbose flag).

New in TOPCAT Full 4.3-1 (Nov 27, 2015)

New in TOPCAT Full 4.3 (Nov 27, 2015)

TAP client enhancements:
The TAP client GUI and implementation in the TAP Load Window have been much improved. As well as being generally more capable and easy to use, the GUI is now quite usable for services providing very large datasets, for instance TAPVizieR (~30,000 tables). In detail:
TAP service discovery is now done by default by requesting search terms relating to the tables, rather than the services, that you're interested in. The old (by service) GUI is still available in a different tab.
The TAP metadata display has been reorganised; schemas and tables are displayed in a searchable tree on the left of the panel, with details of the currently highlighted schema/table in tabs on the right. Organisation of tables at the schema level is now visible. Much more information about the schemas and tables is available, and it's displayed better.
The Columns and Foreign Keys metadata display tables can now be sorted by clicking on column headers.
Much more information about the TAP service is displayed, including a full service description and listing of available geometry functions, user-defined functions and data models (if the service/registry provides this information).
For large services, metadata acquisition is now done as required on a table-by-table basis, rather than attempting to download all the metadata up front. This works much better for services with very many tables.
The query entry panel has many new features, including multiple query entry tabs, full query text Undo/Redo, and limited support for pasting table and column names that have been selected in the GUI.
A new Quick Look mode has been introduced alongside the Synchronous and Asynchronous modes, to execute a query whose result is displayed but not loaded into TOPCAT.
The Examples menu has been extended to include various sub-menus with more ADQL query examples appropriate to the current service, including examples provided by the service itself for services supporting the (somewhat) standard /examples endpoint.
The Examples menu GUI has been modified to facilitate browsing, and a button is provided to link to web pages describing the selected example in more detail.
The TAP menu has new options for configuring Metadata Acquisition, Response Format, Upload Format, Service Discovery, and HTTP-level compression. The old Deletion menu has been moved under this TAP menu as Job Deletion.
Where supported by the service, the BINARY2 VOTable serialization is used by default to retrieve TAP results, which should reduce bandwidth and (marginally) improve handling of null values.
For upload queries, tables are now by default uploaded to the service using the TABLEDATA rather than BINARY VOTable serialization. BINARY ought to work, but at least some services (e.g. CADC) currently fail with BINARY uploads. The default can be changed using the TAP|Upload Format menu item.
There is a change in the way that TAP service metadata is interpreted: table/column names are now expected to be delimited if required. An attempt is made to fix up names that not delimited when they obviously should be. May cause some ADQL examples to work incorrectly in the presence of non-compliant service table metadata reports.
The ADQL syntax highlighting is now sensitive to User-Defined Functions declared by the service (if any are declared), so that misuse of UDFs or use of non-existent UDFs can signalled. This is thanks to an upgrade to Grégory Mantelet's ADQL parser library.
Improve error reporting in the face of non-VOTable TAP error responses. In many cases this makes it much easier to see what's going wrong with a TAP query.
As a diagnostic tool, when making TAP queries, a log message giving a roughly equivalent curl(1) command is now issued at the CONFIG level (visible using flags -verbose -verbose).
Bugfixes:
Fix a serious bug in processing of FITS bit vector (TFORMn='rX') columns. Values read from these columns are presented as a boolean[] array. In all previous versions of STIL the bits have appeared in that array in the wrong sequence (LSB..MSB per byte rather than the other way round). Apologies to anyone who may have got incorrect science results from this error in the past, and thanks to Paul Price for helping to diagnose it.
Fix a less serious bug with TFORMn='rX' processing; attempting to read a single-element bit vector column (TFORMn=1X or X) previously resulted in an error making the file unreadable. Values read from such columns are now presented as Boolean scalars.
Fix a VOTable reading bug relating to bit vector columns (datatype="bit") appearing in BINARY/BINARY2 serializations. This one was more obvious, it would usually generate an error when attempting to read the file.
Fix serious bug in time conversion for CDF TIME_TT2000 data types. This was giving completely wrong (deep future) labels when using TIME_TT2000 columns in time plots.
Fix sky plot vector bug: delta latitude and delta longitude values were being used the wrong way round.
Upgrade JEL to v2.0.2. Fixes problem with evaluating void-typed expressions (especially user-supplied activation actions), and possibly some other obscure bugs.
Prevent submission of multi-cone/SIA queries for which the Search Radius parameter is blank.
Minor enhancements and behaviour changes:
Added some functions to the Arrays class that return array-valued results from array-valued parameters: add, subtract, multiply, divide, reciprocal, condition.
HTTP-level compression is now requested (Accept-Encoding gzip request headers included) in most places where it makes sense. This should substantially reduce required bandwidth in communications with HTTP services that support this.
When clicking on the headers of columns to sort the rows in displayed JTables that show registry services and VizieR catalogues, clicks now cycle between three options (up, down, none) rather than just two (up, down).
Minor improvement to version reporting (reports java specification version, no longer issues warning for absent revision string).
Update JCDF library to v1.1 (minor changes to do with leap seconds).
Now only runs in PLASTIC mode if the -plastic flag is set explicitly (otherwise runs in the default SAMP mode). Previously it ran in PLASTIC mode if a .plastic lockfile was present.
Activation action summary messages are now written via an INFO log message rather than just printing to standard output. To see them you now need to run the application with the -verbose flag. The various exec functions however do now write their output direct to stdout/stderr rather than including them in a return value.
Custom Code activation action invocation behaviour has subtly changed when one of the values in the expression is from an integer or boolean column, and that column has a blank value. In that case, it's not possible to represent the column value properly to the function (since integer/boolean values can't be null). Previously, the function was invoked with a zero/false value where the null would have been. Now, the function is not invoked at all. Note this only applies to Activation Actions (which are expected to have side-effects); expressions evaluated as column values etc are, as previously, executed but the result is ignored.
Modify the heuristics that determine whether the first row of a CSV file is a header.

New in TOPCAT Full 4.2-3 (Nov 27, 2015)

Histogram enhancements:
New Kernel Density Estimate plotting option variants added in the Histogram and Plane plot windows: KDE, KNN and Densogram.
More flexible normalisation options for histogram/KDE plots: normalise by area, maximum bar height, or total bar height.
The Bars fixed control is now relabelled Bins, and has separate tabs for Histogram, KDE and General controls.
The existing Bin Size slider, and new kernel Smoothing sliders, now display current width in data units as the slider is moved.
More bar styles are available for Histogram plots; Semi Filled (the new default) and Semi Steps options make it easier to understand multiple overplotted data sets. Note Semi Steps does not currently export very nicely to PDF/EPS.
Sketch mode is switched off by default.
Minor improvements:
More slider controls have options allowing explicit entry of values from the keyboard (colour map quantisation, marker/vector/ellipse scaling values).
More slider controls have text value readout displays alongside the slider (colour map quantisation, marker size/errorbar scaling values).
Column data read in as unsigned bytes will now be written out as unsigned bytes where the output format permits; previously they were forced to 16-bit signed integers. This affects FITS, VOTable and CDF I/O handlers.
The Subset Styles panel in table data layer controls now has a Visible checkbox indicating whether the subset undergoing subset-specific configuration is currently visible in the plot.
Be less strict about recognising colfits files (tolerate implicit TDIMn headers).
Tables specified on the command line may now use syntax that reads data from a unix pipeline.
OSX application bundle fixed to run application in user's home directory (broken in previous version).
Bug fixes:
Fix minor ADQL parse failure when using TAP tables named without explicit schema prefix by service.
Fix regression bug (from v4.2-2) that caused problems in Histogram and Spectrogram layers: New subset from visible action wasn't working, and the displayed point count was always zero.
Work round FITS read bug that could cause problems for VOTables using inline FITS serialization, and possibly elsewhere.
Fix bug that caused trouble when auto-ranging a plot with a single sky position.
Catch and report the error properly if the user enters a bad explicit axis range (max value

New in TOPCAT Full 4.2-2 (Nov 27, 2015)

Plotting enhancements:
New Linear Fitting form for Plane plot. This reinstates behaviour available from old plot windows, but it provides some more options, like point weighting.
New SizeXY form, which allows plotting (optionally autoscaled) markers with horizontal and vertical extents independently determined by input data.
New Frame fixed control in plot windows with two tabs, Size and Title. Size allows explicit setting of output plot dimensions and alignment in pixels - can be useful when exporting plots. Title lets you add a text title at the top of the plot.
New Save/Import options in Histogram window allow you to export histogram bin data as a new table (this reinstates behaviour available from old plot windows).
More flexibility when assigning colour maps, in the Aux Axis control, Density shading mode, and Spectrogram layer control. These now provide a Scaling option allowing use of Square and Sqrt mappings as well as Linear and Logarithmic, and a Quantise slider allowing quantisation of the colour map to discrete levels.
Rescale Y button added to Histogram window.
Histogram normalisation option adjusted so that total area under bars, rather than total height of bars, is fixed.
Tick Crowding configuration option added to Aux Axis control Ramp tab. Also changed default for Aux axis to use fewer ticks.
Some "dart" options (fixed-base open or filled triangles) added for plotting vectors.
Some "triangle" options (variable-base open or filled triangles) added for plotting ellipses.
Max Marker Size option replaced with with Autoscaling option in plot Size form. You can now optionally turn off autoscaling and specify marker size in pixels instead.
Every slider control in plot windows now has a little button on the right that resets it to its default value.
The text labels for configuration options and coordinate entry fields in the plot windows now provide a tooltip of the form "name=value". These strings give the STILTS parameter assignment corresponding to the current setting.
Other enhancements:
New Column Classification Window lets you add mutually exclusive subsets based on column contents.
FITS I/O:
Reworked part of the FITS input implementation, in particular adjusting the way memory mapping is done to reduce resource requirements on some platforms. If you notice any difference, it should be reduced virtual and perhaps resident memory usage, and some (~10%?) performance improvements, when reading large FITS/colfits files. However, please report anything that appears to be working worse than before.
Colfits files can now be accessed from streams, not just uncompressed disk files (though that's not necessarily a good idea).
MacOS packaging:
Some changes have been made to the MacOS DMG file/application bundle. I believe these are at worst harmless, but pre-release testing of platform-specific features is patchy, so problems are possible - please report if you find any, or if the following items don't seem to be working.
OSX application bundle adjusted so that the DMG file should work on any MacOS with either legacy Apple Java 6 or Oracle/OpenJDK Java 7+ installed.
OSX application bundle adjusted (I think) so that application working directory is user's home directory.
Bugfixes and workarounds:
Fixed a query bug (missing REQUEST=queryData parameter) in the Multiple SSA Query window. This long-standing bug would have stopped this window working at all with well-behaved SSA services.
Fixed error in fits-var output (PCOUNT header card did not include block alignment gap).
Graphics coordinates are now calculated in floating point rather than as integers. This fixes problems that could cause scaled vectors, ellipses etc to be drawn with shapes or orientations badly wrong due to rounding errors. It also improves plotting of analytic functions, especially to vector contexts (PDF/EPS).
Fix some problems to do with zooming to very large/small plot axis ranges.
Hide error bars (etc) that would extend to negative values on logarithmic axes; previously they were being drawn in anomalous places.
Fix Aux axis positioning for 3D plots so that the numeric labels don't get snipped off at top and bottom.
Add a hack that allows LDAC FITS tables to be treated sensibly in auto-format-detection mode.
Make VOTable handling more robust against unknown (illegal) datatypes.

New in TOPCAT Full 4.2-1 (Nov 27, 2015)

New in TOPCAT Full 4.2 (Nov 27, 2015)

Significant new functionality and changes:
Add the new CDS X-Match Upload Window. In most cases (matching with a table that can be found in VizieR) this window can and should be used instead of the old Multi-Cone window - it is much faster. The button for this window replaces the MultiCone button in the main Control Window toolbar. MultiCone is still available from the VO and Joins menus, but is now for most purposes deprecated.
Windows with a Registry Query Panel can now talk to the registry using either RI1.0 (the only option in previous versions) or RegTAP. RegTAP is generally much better (faster, less bandwidth, more reliable), and is the default choice, but a selector allows you to revert to RI1.0 if preferred.
The "frame"-style zooming that was used in old-style plots is now available in the new-style plots - drag the mouse using the middle button or shift key.
The TAP Window button replaces the Concatenate window button on the toolbar. The Concatenate window is still available from the Joins menu. This is not new functionality, but the UI change reflects the growing usefulness of TAP services.
Minor improvements:
Form configuration options are now preserved, rather than reset, when a new table is selected in one of the plot windows.
Registry search panels now display resource count at the bottom of the resource list.
Update the endpoint for the NVO/VAO RI1.0 registry (now deprecated in favour of RegTAP), on advice from Theresa Dower.
Fixed TAP load dialogue so that by default it stays open after a table is loaded (the Stay Open option is set true by default).
Improve TAP window Running Jobs display. The Job URL is now always shown even when the other job details scroll off screen, and the details scroll to the top not the bottom when a job is selected or refreshed.
Upgraded Gregory Mantelet's ADQL library to version 1.2. Better ADQL parsing.
Modify the way antialiasing is controlled in plot windows. The Grid antialiasing option, where present, now just covers the actual grid lines. Text antialiasing is now selected with a new "Antialias" option in the (now possibly misnamed) "Text Syntax" item in Font configuration. This now affects the Aux axis and Legend labelling as well as the grid labelling.
Added new graphics output format png-transp to generate PNG files with transparent backgrounds.
Bug fixes:
Set antialiasing on by default for OSX - get it right this time.
Fixed some bugs related to TAP table uploads.
Fix out-by-one error in reporting of Row of min/max values in Statistics window.
Add missing geometric reserved words to ADQL reserved word list. Has a slight effect on ADQL validation in TAP window.
No longer disable "View URL As ..." options in Activation window when no suitable columns are present.
Fixed bug in which cube plot navigation help button gave you sphere plot help and vice versa.

New in TOPCAT Full 4.1 (Nov 27, 2015)

In this version, the new plotting windows appear for the first time in the main Control Window toolbar, and are no longer considered experimental. There have been many changes to their user interface and functionality since the last release. The old plotting windows are still available from the Control Window Graphics menu, but are now considered somewhat deprecated, and will not be developed or fixed further.
Navigation changes to the new plotting windows:
In 2d plots, right-button drag (or ctrl-drag) does a 2-d anisotropic zoom - you can stretch or squash the plot in both directions.
In 3d plots, right-button drag (or ctrl-drag) does a 2-d anisotropic zoom along the 2 axes most face-on.
In 3d plots, middle-button drag (or shift-drag) slides the plot along the 2 axes most face-on.
In 2d plots, the pan/zoom actions (wheel, pan-drag, zoom-drag) can be done outside the plot to affect just one axis.
Most navigation mouse actions now show visual feedback on the screen.
A new tab Navigation is added to the Axis controls. This lets you configure zoom/pan actions, for instance selecting zoom factor and X/Y/XY zoom options for when mouse wheel is used.
A hints panel is added below the plot showing current mouse navigation actions.
Removed zoom in/out buttons in the toolbar of the layer plot windows; there are better ways to zoom.
Other changes to the new plotting windows:
Histogram plot type added. You can also add histograms to a normal plane plot if you want to.
Layer plot windows now have a progress bar at the bottom for loading data, and optionally for other potentially slow operations like replotting, identifying a point, turning blob-selected points into a subset etc.
For very large/slow plots, intermediate subsampled plots are optionally displayed while panning/zooming to improve responsiveness.
Toolbar buttons rearranged; the buttons for adding and removing layer controls are now in the layer control panel not the main plot window toolbar.
When a pair crossmatch completes successfully, you get the option to plot the results, which shows you exactly how the located matches relate to the input positions (pairs of points are plotted).
Text label plotting now has crowding configuration options: according to configuration, text labels are only drawn if there aren't too many close together.
New grid and label colour configuration options for plane and sky plots.
More colour maps added.
Antialiasing option added for grid line and label drawing.
New auto transparency shading mode.
Add some padding around auto-ranged plot limits.
Changing table in a plot now tries to retain the coordinate values if they still make sense.
Axis tick labelling improved, including avoiding overlapping tick labels.
Sky plot re-ranging improved: new data set now re-ranges unless lock button is set.
Clip selector added for density and aux colour maps, which means you can now use just part of a colour map, giving you much more control over colouring.
Sky plot position formatting now honours sexagesimal setting.
Some minor threading/performance improvements.
Experimental time plot type added. This plot type is considered experimental and appears in the Graphics menu but not currently on the main toolbar. It can plot time series and spectrograms. May be improved in future versions.
The control for plotting pairs of points works differently (now has its own layer control not just a Form). The triple-point form (which didn't work properly) is withdrawn for now.
Other enhancements:
A new experimental SAMP message table.get.stil is supported, allowing SAMP clients to retrieve tables from TOPCAT.
URL selector fields in Activation Action window are now editable.
Add cuboid match algorithm to match windows.
The Exact matcher now considers scalar numeric values equal if they have the same numeric value; they are no longer required to have the same type.
Array-valued per-column metadata items are now displayed properly.
The toString function now works for byte and boolean values as well as other data types.
Replace the (slightly dangerous, also undocumented) Apply Subset option in the table viewer window with a new Highlight Subset option. Also add a new Highlight Subset option in the Subsets window.
In the old-style histogram plot, exporting it now provides cumulative counts if the visible histogram is currently in cumulative mode.
Keyboard focus is more helpfully positioned by default in some dialogue windows (it starts off in the first text field).
Bugfixes etc:
Fix a registry access bug related to namespaces. It is now finally possible to query the STSci/NVO/VAO registry, as well as the AstroGrid and Euro-VO registries, from the Cone/SIA/SSA/TAP VO windows. Queries to the STSci registry have not been working since mid-2010 as a consequence of bugs in (mostly) the TOPCAT client and (partially) the STSci registry service.
Work around bug in MacOS text rendering in plots that painted text in the wrong direction when not horizontal (e.g. letters going backwards when running vertically). The workaround involves defaulting to antialiased text when drawing axis grids for MacOS. As well as getting the text the right way round this does look a bit nicer, but it is also perceptibly slower. If speed is more important than correctness you can turn it off using the Antialias checkbox in the Axes-Grid tabs of the various plot windows.
Withdraw Labels tab in the Sphere plot, since it didn't do anything.
Rename the File menu in most windows to Window, which is much more appropriate.
Fix bug when concatenating tables whose data cells may have been edited by hand.
Fix service selection bug (NullPointerException) in multi-SIA/SSA windows.
Fix bug which prevented access to long integer array elements from expression language.
Fix bug in Statistics window so that Boolean columns correctly report mean values (true/true+false proportion).
Fix TST input handler so TST files with fewer than 3 columns can be read.
Upgrade JSAMP to version 1.3-4 (hub now supports web clients with https URLs).
Moved text documenting table.load.cdf and table.load.stil SAMP MTypes from the transmit section (wrong) to the receive section (right) of this document.
Minor improvements to CDF support.
Fix some plot threading issues.
The reported point count is improved in plots.
Better handling of OutOfMemoryErrors during plotting.
Better handling of LaTeX errors in axis labelling.

New in TOPCAT Full 4.0-1 (Nov 27, 2015)

New Functionality:
Add read-only support for CDF (NASA Common Data Format) files. Currently, no attempt is made to present time-like values in a human-friendly way, but this may improve in a future release. The SAMP table.load.cdf MType is also supported for receiving CDF files.
Improve handling of HTTP basic authorization. HTTP 401s now pop up a user/password dialogue window, unless the star.basicauth.* system properties have been set up.
Add options to calculate the (scaled or unscaled) Median Absolute Deviation in the Statistics window.
Minor enhancements:
Now subscribes to SAMP voresource.loadlist.tap MType, accepting resource list in TAP load dialogue "Select Service" tab.
The Columns window now makes a better attempt at displaying non-standard per-column metadata of unusual types, in particular array values.
Line plotting improved in layer plots (including better dashing, new antialias option).
Implemented fixes to reduce the chance of users inadvertently overloading external Cone/SIA/SSA services with multicone-like queries. First, fix it so that abandoned queries are properly terminated, rather than continuing to hit the server until completion or JVM shutdown. Second, implement a sensible default maximum value for the Parallelism field in the multi-cone (etc) panel (though this may be adjusted with a system property).
Source code is now managed by git and not subversion. The format of the "Starjava revision" string reported by the -version flag and the Help|About menu item has changed accordingly.
Bug fixes:
Fix CSV regression bug introduced at v4.0b - CSV files now work again with MSDOS-style line breaks.
TAP example queries now quote table and column names where necessary. This fixes a bug that was particularly evident with the VizieR TAP service for which nearly all table names are not legal ADQL identifiers.
Quoting behaviour has changed when generating SQL to write to RDBMS tables. This ought to reduce problems related to mixed-case identifiers. However, it is possible that it could lead to unforseen new anomalies.
Fix bug with reading session files containing 1-column tables.
Fixed FITS output bug which could result in badly-formed string-valued header cards (no closing quote).
Turn off layer plot optimisation that could result in lost precision for double values.
Fix layer plot bug that broke the plot if both handles of a range slider were dragged to the same position.
Fixed it so that layer plot axis range settings are reset when the Rescale button is used.
Fixed a bug that caused an exception when a table in a layer plot was deleted from the application.
Fixed a bug when zooming way out from a sky plot.

New in TOPCAT Full 3.9 (Jul 5, 2012)

Crossmatching:
Additional asymmetric match options have been added to the Pair Match window - as well as the old symmetric Best Match option, you can now choose to do a match which identifies the best match for each row of one or the other input table (see Output Rows Selector Box). They correspond to finding the best match in table B for each row in table A, and in crowded fields often provide more intuitive semantics than the previous symmetric best option (in non-crowded fields there is generally no difference). This replicates the matching performed by some other tools, including Aladin.
New Match Algorithms, 2-d Cartesian Ellipses and Sky Ellipses, have been introduced to permit matching of general elliptical, rather than just circular, regions in both planar and sky coordinates. Another, N-d Cartesian with Errors, has been added for dealing with per-object errors in Cartesian coordinates (previously per-object errors could only be handled in sky coords).
Fixed a significant bug in sky crossmatching. If all points in a table were on one side of the RA=0 line, but the error radius extended across that line, matches on the other side could be missed. Matches could also be missed if different tables used different conventional ranges for RA (e.g. -180..180 in one case and 0..360 in another). This fix may in some, but not most, cases result in slower matching than previously.
Semantics of the Sky With Errors match algorithm have changed slightly.
Notable usability improvements:
You can now change the column order of a table by dragging the rows up and down in the Columns Window.
Algebraic functions involving angles are now mostly available using degrees as well as radians. The Coords class has been replaced by CoordsDegrees and CoordsRadians classes providing sky coordinate functions, and a new class TrigDegrees provides normal degree-based trigonometric functions alongside the radian-based versions in Maths. The functions in the various activation action classes now take degrees and not radians. Some of the old function names have changed to make clear that they use radians and not degrees.
The File|Discard Table action in the Control Window can now remove multiple tables at once if more than one is selected.
ADQL parsing and syntax highligting has been added to the TAP load dialogue, thanks to Grégory Mantelet's ADQL library.
TAP queries are now synchronous by default (the Synchronous checkbox is now checked by default).
The SQL Query Dialogue (finally) has a multi-line entry field for SQL query text.
Add To Browser button to Help Window - this displays the currently displayed help page in your normal web browser.
New convenience button added to Available Functions window which is a shortcut to display of the expression syntax in the Help browser.
Other upgrades and enhancements:
JSAMP has been upgraded to version 1.3-1. The main change is that the Web Profile is now enabled by default in the SAMP hub that TOPCAT launches. There are also more options in the Hub menus, and it will overwrite any moribund .samp file at startup, which should eliminate annoying persistent "404 No handler for URL" warnings.
Add Delete On Exit checkbox to Running Jobs tab of TAP load dialogue. Jobs are now created by default with Delete On Exit set true (you can change the default with the Deletion menu).
You can now run JyStilts from the topcat-*.jar jar files in the same way as from stilts.jar.
Add experimental system properties star.basicauth.user and star.basicauth.password.
Added the experimental topcat.exttools system property to allow custom tools to be added to the main toolbar at runtime.
Improve resilience of multi-cone operation in the presence of unreliable or inconsistent DAL services.
Added new constants to expression language POSITIVE_INFINITY, NEGATIVE_INFINITY and NaN.
Add -running flag which loads tables specified on the command line into a existing instance of TOPCAT if one is already running.
Add new join function to Arrays class to combine array elements into a string.
TAP load dialogue uses modified upload ID as per most recent TAPRegExt draft. This may cause some TAP services incorrectly to appear not to support uploads if they have not made a similar update.
Bug fixes:
Improved some issues (reporting wrong row subset counts and membership) related to changing subset definitions.
Fix cone search verbosity parameter so that VERB=3 is not erroneously ignored.
Fix bug introduced at v3.7 related to labelling loaded files (improper handling and propagation of LOAD_SOURCE table parameter).
A PARAMref element with no referent in a VOTable no longer causes an uncaught NullPointerException.
Fix a small bug related to enabledness of buttons in control window and hub connection.
Work round an obscure java misfeature which could cause the wrong cell to be edited if a sort interrupts an edit.

New in TOPCAT Full 3.8 (Jul 5, 2012)

TAP;
A new TAP load dialogue supporting data retrieval using the Table Access Protocol (TAP) has been introduced. This provides SQL-like free-form access to any remote databases exposed with the TAP protocol.
SAMP changes:
An internal SAMP hub is now by default started when TOPCAT starts up if no existing hub is apparently running, so under normal circumstances the SAMP panel at the bottom right of the Control Window should be active all the time. This means that you don't need to do any explicit preparation to communicate with other SAMP-aware applications. If you want to return to the previous behaviour (hub only started by explicit user action), use the -nohub flag on the command line. You can connect and disconnect with the hub by clicking the connection logo at the right of the SAMP panel.
The JSAMP library is upgraded to JSAMP v1.2, which fixes one or two bugs and supports the experimental Web Profile, though Web Profile support is not switched on by default.
Other enhancements:
Random Groups HDUs are now tolerated, though not interpreted, within FITS files.
Add quintuple match option to multi-table match window (someone requested it!).
You can now re-activate a row in the table viewer window by deselecting and then re-selecting it.
Expressions in column selectors are now left (and coloured grey) rather than deleted in the case of a syntax error.
Improve text rendering in Available Functions window.
GAVO load dialogue now uses fixed-width font for SQL entry.
Registry search now includes Publisher field by default.
Bug fixes and workarounds;
JDBC table input handler now effectively downcasts BigInteger/BigDecimal types to Long/Double. The PostgreSQL JDBC driver seems to use the Big* types routinely for numeric values (which I don't think it used to do).
Add workaround for J2SE bug #4795134, which could cause errors when reading compressed FITS files.
Fix FITS character handling bug which could cause corrupted FITS files on output in presence of non-ASCII characters.
Fix (some) JDBC connection leaks.
Attempt case-sensitive matching before case-insensitive for column names.

New in TOPCAT Full 3.7 (Jul 5, 2012)

Load Dialogues:
Table loading has had a major overhaul. Load dialogues are no longer modal, that is you can now interact with the rest of the application while they are open.
This has a number of benefits:
A slow load doesn't prevent you from doing other things.
You can be loading several different tables at once, either from the same or from different places.
You can use the Help system while a load dialogue is open (load dialogues now have the usual Help button like other TOPCAT windows).
It has also enabled some related changes:
Progress is reported for each table load - you can see how many rows have loaded and, if known, how many there are in total
Cancelling a load in progress is more straightforward and works more reliably than in earlier versions
A few related bugs and idiosyncracies have been corrected at the same time.
Registry Queries:
The registry search interface used in single and multiple VO access windows has been improved:
When specifying a keyword search for Cone, SIA or SSA services, you can select which registry record fields to match (ShortName, ID, Title, Publisher, Subjects etc). This makes it much easier to get the records you're interested in.
The Subjects field is now displayed for retrieved records.
Lists of registry records (i.e. Cone, SIA or SSA services) can now be sent to/received from other tools via SAMP (voresource.loadlist{,.cone,.siap,.ssap} MTypes).
Sky coordinates can be received from other tools via SAMP and used to set Cone, SIA or SSA position (coord.pointAt.sky MType).
BaSTI Load Dialogue:
A new load dialogue is available for accessing the INAF-OATS BaSTI (Bag of Stellar Tracks and Isochrones) service -- code contributed by Marco Molinaro.
Searchable Help:
The Help browser now has a Search tab as well as the Table of Contents tab, so you can do text searching on the extensive help document.
Minor Enhancements:
Storage management improvements; removed restriction on large (>2Gb) non-FITS datasets in some circumstances.
Fix so FITS tables >2Gb can be used in 32-bit mode (though slower than 64-bit).
All windows have a new Scrollable option in their File menu. It is not generally recommended to use this option, since in general the windows are arranged so that resizing them will resize sensible parts of them, but it may be useful if using some of the larger windows on an unusually small screen.
FITS files now store table names in EXTNAME (and possibly EXTVAR) header cards.
Window placement should now behave a bit more like platform norms, rather than sitting in the top left corner.
When a table is discarded, it is now deselected from any table selector (for instance in a plot, match or concatenation window).
HTML table output now writes cell contents which look like URLs in HTML tags.
Basic authorization (http://user:pass@host/path) on table URLs handled.
Logs current version and whether it is up to date with latest release on startup. This behaviour can be controlled by the [-no]checkversion flag.
Belatedly added STFC logo to About window.
Add recommendation to use 64-bit java in large table tips section.
Bug Fixes:
Fix bug in registry result table which displayed resource identifier instead of publisher in the Publisher column.
Fix SAMP table load bug which tripped over "+" characters in URLs.
Fix regression bug from v3.6 - loading jdbc tables from command line or text entry now works again.
Fixed file pointer int overflow bug in FITS MultiMappedFiles.
Fixed a couple of bugs relating to VOTable strict mode.
Possibly fixed a not-well-characterised bug in registry search.

New in TOPCAT Full 3.6 (Jul 5, 2012)

Load/Save Dialogue improvements:
There is a new System Browser option when loading and saving. This does essentially same job as the old Filestore Browser (still available), selecting a file to load or save, but it uses the system-default GUI to do it. For Mac and Windows users, this may present a more familiar and functional interface to the file system (for Linux users, it is probably not an improvement). Both options are available, so you can choose whichever you prefer.
You can now save multiple tables at once. The Save Window now offers three ways of saving: Current Table, Multiple Tables and Session. Current Table saves the current Apparent Table as before. Multiple Table saves some or all of the loaded Apparent Tables to a container file (typically Multi-Extension FITS or multi-TABLE VOTable). Session saves some or all tables as well, but additionally saves many aspects of the TOPCAT state, such as table subsets, sort order, hidden columns etc.
Other significant enhancements:
Finally - you can delete and rename row subsets in the Subset Window! Apologies to those who've been asking that this has taken so long to implement.
There are new actions in the Control Window File menu to change the order of loaded tables in the table list (move them up and down). You can use ALT-up/down keys to do the same thing. This may be useful in conjunction with the new session/multi-table save facility.
Utype metadata items are now visible in the Parameter and Column windows.
The JSAMP library has been upgraded to version 1.1. This means that the hub will appear in the "System Tray" where possible (when using java 1.6+, and when using a suitable display manager). It also recognises the SAMP_HUB environment variable for non-standard hub locations.
There are a couple of improvements in the VizieR load dialogue: it has options to display sub-tables within catalogues and to include or exclude obsolete catalogues.
Minor enhancements:
The current table can now be discarded by hitting the Delete key in the Control Window table list.
SDSS image cutout downloads updated to DR7 (was DR4).
The unofficial column type "long" is recognised in IPAC format tables.
An efficiency warning is now issued for large compressed FITS files.
Format row counts are displayed with group separators (e.g. thousands separated by commas) in some places for better readability.
Fixes for bugs and misfeatures:
Table parameter values in algebraic expressions now evaluate to their current value, rather than the value when the expression was entered.
Tables in TOPCAT now have private copies of column and table metadata. This means that changing a column name or parameter in one table will not affect other tables. It should also prevent a related bug that allowed the column headings in the table browser to get out of step with those in the columns window and elsewhere.
Receipt of a SAMP table.highlight.row message will no longer cause a similar message to be sent back (and similarly for the PLASTIC ivo://votech.org/votable/highlightObject message). The previous behaviour was an unwanted implementation side effect which could sometimes cause problems in interaction with other tools.
Fixed bug in FITS-plus metadata output (table parameters were getting lost).
Better behaviour (warn + failover) when attempting to read large files on 32-bit OS or JVM.
Corrected literature references in Fluxes conversion class documentation (thanks to Mattia Vaccari).
Fix memory usage issues (unnecessarily large output graphics files, possible out of memory errors) for density plots with large pixel sizes.
Fixed bug in CSV file parsing that could ignore header row in absence of non-numeric columns.
Fix some minor bugs with Cancel operation during table save.
Reinstate help buttons from Save Window toolbar (erroneously removed since version 2.0).
Make sure that failed load of table with no rows reports as such rather than failing silently.
Loading tables by typing their name directly in the Load Window now loads multiple tables if present not just the first one, in common with most of the other load dialogue types.
Upgraded PixTools HEALPix library to 2010/02/09 version. This fixes a bug that could theoretically cause deficient crossmatch results, though I haven't managed to produce such errors.
Fixed (I hope) "Table no longer loaded" error when sending tables via SAMP.
Finally, from this release TOPCAT requires version 1.5 (a.k.a. 5.0) of the Java J2SE Runtime Environment; it will no longer run on version 1.4, which is now very old. I don't expect this to cause compatibility issues for anyone, but I'm interested to hear if that's not the case.

New in TOPCAT Full 3.5-2 (Jul 5, 2012)

Add memory monitor indicator in control window.
Considerable performance and scalability improvements to the crossmatching algorithms. For several common regimes, using default settings, memory use has been decreased by a factor of about 5, and CPU time reduced by a factor of about 3.
Add optional tuning controls (Tuning Parameters and Full Profiling) to the crossmatching windows. Experimentation with these can lead to significant performance improvements for given matches.
Fixed a crossmatch bug; it was giving a possibility of suboptimal "Best Match Only" match assignments when pair matching in crowded fields. Crossmatch results thus may differ between earlier versions and this one. Both are reasonable, but the newer behaviour is more correct. In non-crowded fields, there should be no change.
Add new functions to Arrays: array functions for constructing arrays, and new aggregating functions median and quantile.
Added Verbosity selector in multi-cone window.
Improved logging of registry searches.
Adjusted the way that data types are read from JDBC databases. Date, Time and Timestamp type columns will now be converted to Strings which means they can be written to most output formats (previously they were omitted from output tables).
Match Window documentation extended and somewhat reorganised (they now have their own appendix subsection).
MacOS icons improved: thanks to Simon Murphy (ANU) for this contribution.
Memory management adjusted further - default (Adaptive) storage policy now uses direct allocation (=malloc()) for intermediate-sized buffers to avoid running out of java heap space.
Further performance improvement for VOTable documents with very many TABLEs.
Fix bug in code for handling very large mapped FITS files. This was causing fatal read errors in some cases.
Fix issue when sending some SAMP messages (unhelpful reuse of message tags).
Fixed minor error when a directory in the file browser got deleted while visible.
Fix minor scrolling bug in table and column windows.
Fix histogram save/export bug: the exported table now has the same X bounds as the currently displayed histogram.
Withdraw embedded SPLAT application. SPLAT could previously be used in some circumstances for in-JVM display of spectra, but this facility didn't work well and was presumably little used. You can still view spectra by configuring an Activation Action to talk to an external spectrum viewer (SPLAT or some other choice) via SAMP or PLASTIC.
Withdraw SOAP server functionality. This has been deprecated since v2.1. External control is better handled using SAMP (or PLASTIC). If anybody misses this functionality, get in touch and I will consider a similar but less SOAPy alternative.

New in TOPCAT Full 3.5-1 (Jul 5, 2012)

New in TOPCAT Full 3.5 (Jul 5, 2012)

There are several improvements in this version, many of them to do with improved functionality and usability for loading or acquiring data both from local files and from various Virtual Observatory services.
Multiple-table load
TOPCAT will now load multiple tables from a single file if multiple tables are present, rather than just the first one. This applies to multi-extension FITS files which contain more than one table HDU, and to VOTable documents which contain more than one TABLE element. It affects the Filestore dialogue, naming files on the command line, and the new VizieR load dialogue.
In previous versions, when you selected a multi-extension FITS file or VOTable from one of the load dialogues or on the command line, only the first table contained in it would be loaded, and any others were silently ignored. Such files often only contain a single table, but not always. Now, if multiple tables are present in the container file, each one will appear as a new table in the tables list. Other table file formats are not currently affected, since there is no mechanism for storing multiple tables in, say, a CSV file.
Multiple Cone, SIA and SSA searches
In previous versions, it was possible to use the Virtual Observatory Cone Search, Simple Image Access and Simple Spectral Access services to locate catalogue records, images and spectra respectively in a single (cone-shaped) region of the sky, with position and radius entered manually. In this version it is possible to execute such a query for each row of an input table, with position and radius obtained from table data in appropriate columns. This effectively gives a join between a local table and a remote one exposed by a cone-like searchable VO interface.
VizieR load dialogue
A new VizieR load dialogue has been introduced to make queries directly to CDS's VizieR service. Thanks to Thomas Boch for contributing code for this.
VO menu
A new menu labelled VO (for Virtual Observatory) has been added to the Control Window. This currently contains single and multiple Cone, SIA and SSA queries and the service specific load dialogues, namely the VizieR and Millennium ones. All these features are also available elsewhere in the GUI, but the new menu groups together VO-related functionality for convenience.
Positional search dialogue enhancements
A number of improvements have been made to the Cone Search, SIA and SSA dialogues; these applies both to the old single-region search dialogues and the new multi-region searches.
The service URL can now be entered (e.g. cut'n'pasted) directly as an alternative to obtaining it from a registry search. This allows use of unregistered services, as well as making it easy to select the service URL for pasting elsewhere.
The resource table returned from the registry search can now be sorted by column contents, by clicking on the selected column header.
While searching the registry, some indication of the number of services found so far is displayed.
Searching the registry is less likely to result in an out of memory error.
The (not very useful, and potentially confusing) registry Update button has been withdrawn; its function is now available only from the Registry menu instead.
The metadata contained in tables returned from a successful search is improved.
The SIA and SSA dialogues now have a selector for choosing the desired image/spectrum format.
The SIA and SSA dialogues now longer require explicit entry of a size parameter (these protocols define default behaviour for when no non-zero size has been specified).
The SIA and SSA now automatically query the registry for all appropriate services when first displayed. This is sensible behaviour at the moment, since there is at time of writing a reasonable number of them. A directed search may be given instead at the user's option.
The Cone Search window provides better instructions on use when first displayed.
A new section Section 6 has been added to the manual providing an introduction to the concepts of VO data access, and a new apendix section Appendix A.8 describes the various single- and multiple-query windows.
Storage Policy change
The way that TOPCAT stores large tables has changed. In previous versions, table data was held in memory (the default Storage Policy was "memory")) unless the -disk flag was specified (to use the "disk" storage policy), in which case it was stored in temporary disk files. If you didn't put -disk on the command line, it was common to run out of memory when working with large tables.
Now, the default storage policy is "adaptive". This means that relatively small tables are held in memory, and larger ones are stored in temporary files. The effect should be that most of the time, you don't need to specify any special options, and you can work with small or large tables without running out of memory. Using -disk is still permissible, and may help in unusual circumstances. Running out of memory is still a possibility, but should be much less common.
The old behaviour can be restored by using the new -memory command-line flag.
The only differences you notice from this change should be good ones, however it is slightly experimental. Please contact the author if you suspect it has introduced any problems.
SAMP-related enhancements
Spectrum Display activation action now uses SAMP or PLASTIC to display in a suitably compliant external spectrum viewer. The old behaviour was to display the spectrum in an internal SPLAT window, but it didn't work very well.
The JSAMP library has been upgraded to v1.0, giving some SAMP behaviour changes, enhancements and bugfixes. Control of things like TOPCAT server port and server endpoint hostname are now configurable using jsamp.* system properties.
The -jsamp command-line flag has been added for convenience so that the TOPCAT jar file can be used easily as a JSAMP toolkit.
VOTable-related enhancements
Namespacing of VOTable documents made more intelligent, and configurable using the votable.namespacing system property.
VOTable 1.2 supported.
The VOTable 1.2 xtype attribute is now used to try to identify columns containing ISO-8601 format dates, which allows them to be treated as numeric values for plotting etc.
As described above, when loading a VOTable document which contains multiple TABLE elements, all the contained TABLEs are now separately loaded into TOPCAT rather than just the first one, which is what used to happen.
Loading changes
The Load Window now has toolbar buttons as well as items in the DataSources menu for more convenient access to the various load dialogues.
The command-line flags corresponding to load dialogues (-tree, -file, -sql, -cone, -gavo, -registry, -sia, -ssa) have been withdrawn from use; they were probably not widely used, and are of minimal usefulness with the new load window toolbar buttons.
While a table is loading an entry is now visible in the Control Window table list. Previously it was not always clear whether a slow load was in progress or not.
Bugfixes and minor enhancements
There are improvements (and possibly new problems) in the loading dialogue framework. One item is that a failed load less often closes the dialogue.
HTML table output is now HTML 4.01 by default (includes THEAD and TBODY tags).
There is a workaround for illegally truncated type declarations in IPAC tables.
XML, including VOTable, output is handled more carefully; fix VOTable output encoding to be UTF-8, and ensure no illegal XML characters are written.
Fixed plotting bugs - caused intermittent GUI corruption (e.g. missing toolbar buttons), especially on MacOS.
Bug fixed in crossmatching output: entries which should have been null were sometimes written as non-null (typically large negative numbers) in FITS and in non-TABLEDATA VOTable output. This affected cells in otherwise non-nullable columns where the entire row was absent. The previous behaviour is not likely to have been mistaken for genuine results.
parse* string->numeric conversion functions now cope with leading or trailing whitespace.
The list of example queries in the GAVO load dialogue has been extended to match those in its web interface.
Fixed an obscure bug which could under rare circumstances cause truncation of strings with leading/trailing whitespace read from text-format files.

TOPCAT Full Changelog

What's new in TOPCAT Full 4.9

New in TOPCAT Full 4.8 (Jan 12, 2021)

New in TOPCAT Full 4.7 (Nov 19, 2019)

New in TOPCAT Full 4.6 (Apr 26, 2018)

New in TOPCAT Full 4.5-1 (Nov 8, 2017)

New in TOPCAT Full 4.5 (Oct 2, 2017)

New in TOPCAT Full 4.4 (Mar 27, 2017)

New in TOPCAT Full 4.3-5 (Oct 12, 2016)

New in TOPCAT Full 4.3-4 (Oct 12, 2016)

New in TOPCAT Full 4.3-2 (Nov 27, 2015)

New in TOPCAT Full 4.3-1 (Nov 27, 2015)

New in TOPCAT Full 4.3 (Nov 27, 2015)

New in TOPCAT Full 4.2-3 (Nov 27, 2015)

New in TOPCAT Full 4.2-2 (Nov 27, 2015)

New in TOPCAT Full 4.2-1 (Nov 27, 2015)

New in TOPCAT Full 4.2 (Nov 27, 2015)

New in TOPCAT Full 4.1 (Nov 27, 2015)

New in TOPCAT Full 4.0-1 (Nov 27, 2015)

New in TOPCAT Full 3.9 (Jul 5, 2012)

New in TOPCAT Full 3.8 (Jul 5, 2012)

New in TOPCAT Full 3.7 (Jul 5, 2012)

New in TOPCAT Full 3.6 (Jul 5, 2012)

New in TOPCAT Full 3.5-2 (Jul 5, 2012)

New in TOPCAT Full 3.5-1 (Jul 5, 2012)

New in TOPCAT Full 3.5 (Jul 5, 2012)

New in TOPCAT Full 3.4-3 (Jul 5, 2012)