Softpedia >Windows >Programming >Debuggers/Decompilers/Disassemblers >OpenRefine >  Changelog

OpenRefine Changelog

What's new in OpenRefine 3.8.0

Apr 29, 2024
  • Improve new release notification for better flexibility and privacy (#6511) by @wetneb

New in OpenRefine 3.8 Beta 1 (Feb 22, 2024)

  • New features and improvements:
  • Keyboard navigation improvements:
  • Dialogs are focusable (#5578, by @Abbe98)
  • The tab order for reconciliation create and match buttons is fixed (#5685, by @Abbe98)
  • The show/hide left panel is keyboard accessible (#5852, by @Abbe98)
  • Menu buttons in the extension-bar can be opened with the keyboard (#5853, by @Abbe98)
  • Tab buttons are outlined when focused (#5851, by @Abbe98)
  • The element outlines set by the browser are retained (#5867, by @Abbe98)
  • The project permalink can be selected (#5871, by @Abbe98)
  • The button to rename a project can be selected (#5868, by @Abbe98)
  • The column header buttons are selectable/openable by keyboard (#5854, by @Abbe98)
  • Make action area tabs selectable by keyboard (#5885, by @Abbe98)
  • Make it possible to set custom headers using only the keyboard when fetching urls (#5886, by @Abbe98)
  • The menu system is navigable by keyboard (#5901, by @Abbe98)
  • The "Get data from" menu is keyboard accessible (#5900, by @Abbe98)
  • Cells in the grid can be edited by keyboard (#5855, by @Abbe98)
  • Reconciliation usability improvements:
  • The waiting screen while guessing reconciliation types is internal to the reconciliation dialog (#4877, by @ayushrai206)
  • The "auto-match" checkbox persists after restarts of OpenRefine (#4722, by @ayushrai206)
  • The documentation of the reconciliation service is displayed in the reconciliation dialog (if available) (#5784, by @ayushriai206)
  • The default types supplied by the reconciliation service are always offered to users (#4224, by @ayushrai206)
  • The reconciliation types are displayed with both name and id (#5907, by @ayushrai206)
  • OpenRefine honours the batch size announced by reconciliation services (#5603, by @ayushrai206)
  • The dialog for the operation to add a column of entity identifiers is improved (#5998, by @elebitzero)
  • Errors encountered by the reconciliation operation are displayed in the grid and are available via the cell.recon.error GREL expression (#3194, by @ayushrai206)
  • Those errors can also be isolated via facets (#6232, @ayushrai206)
  • The interface to select a reconciliation service when reconciling was improved (#6118, by @ayushrai206 and @Lydiaofficial)
  • The "Search for match" option is present in cells with reconciliation errors so that they can be fixed manually (#6192, by @ayushrai206)
  • The error messages generated during reconciling are more helpful (#6111, by @ayushrai206)
  • A new operation to extract URLs for reconciled cells is available (#5960, by @ayushrai206)
  • Property selection in the reconciliation dialog gives better feedback to the user about whether a column is successfully mapped to a property or not (#6060, by @elebitzero)
  • Type selection is similarly improved (#6131, by @elebitzero)
  • Only up to three reconciliation candidates are displayed by default, with the option to see more (#6154, by @ayushrai206 and @Lydiaofficial)
  • A new design for the reconciliation dialog was proposed but has not been implemented yet. Your opinion about it is welcome on the forum. By @Lydiaofficial
  • It is possible to discover the source of a column obtained by fetching data from a reconciliation service, by hovering the column header (#5130, by @ayushrai206)
  • Facet improvements:
  • The height of text facets is fitted to their contents (#5514, by @elebitzero)
  • The text facet windows to their content (#5513, by @elebitzero)
  • The facets panel does not shift vertically on refresh (#5610, by @elebitzero)
  • Improvements to linking to specific parts of OpenRefine via URLs:
  • It is possible to link to a given home screen panel (#5597, by @Abbe98)
  • The tags used to filter projects are also reflected in the URL (#5769, by @Abbe98)
  • Layout improvements to the Wikibase extension:
  • Layout improvements in the Wikibase schema editor fixes (#5449, by @prashasti-7)
  • The Wikibase issues tab displays the number of schema validation errors (#5358, by @SoniaSun810)
  • The list of available Wikibase schemas is numbered (#5893, by @Hisiste)
  • Wikibase: Layout and label changes to schema alignment tab (#5346, by @Abbe98 and @lozanaross)
  • Clicking on the triangle unfolds Wikibase references in the schema editor (#6304, by @amparab and @cooperzoe)
  • Other improvements:
  • Applying a list of operations stored in a JSON file is possible without copy and paste (#5022 by @IjayAbby)
  • The cluster choice limit is configurable via the preferences (#5847, by @5tigerjelly)
  • The metadata dialog shows JSON contents in a more readable way (#5870, by @Abbe98)
  • OpenRefine is known to be compatible with Java versions 11 to 20 (#5930, by @wetneb)
  • URIs with the geo: protocol are rendered as links in cells (#5940, by @Abbe98)
  • Project archives can be imported via URL (#5431, by @SoniaSun810)
  • A Windows installer is available on top of the existing zip distribution. We no longer publish zip distribution without embedded Java. Do let us know if this is a problem for you. (#3224, by @dori4n)
  • CSV Byte Order Mark (BOM) is supported (#1241, by @tfmorris)
  • More HTTP headers are supported when fetching a column via URLs (#6334, by @tfmorris)
  • Columns can be expanded with a more easily identifiable button (#5879, by @VhugoJc)
  • TSV export avoids adding unneccessary quotes around cells (#2071, by @tfmorris)
  • GREL changes:
  • The forRange GREL function accepts negative increments (#5520, by @Huishin-pie)
  • Accessing the record & columnNames fields works again (#5633, by @tfmorris)
  • The split() function filters trailing empty token when there is a trailing string separator and leading empty token when there is a leading pattern separator match (#5587, by @tfmorris)
  • The replaceEach() function is more faithful to its documentation (#5463, by @Huishin-pie)
  • The cross() function returns an empty list on no match (#5531, by @jenny-Musah)
  • The length() function returns the number of keys in object (#5991, by @tfmorris)
  • The forEachIndex() control supports JSON objects and arrays (#3147, by @tfmorris)
  • Controls (+ and comparison operators such as <) have less unexpected behaviors

New in OpenRefine 3.7.9 (Feb 11, 2024)

  • Vulnerabilities:
  • (fixed since 3.7.8) The fix for the MySQL vulnerabilities CVE-2023-41886 and CVE-2023-41887 was corrected, as it still allowed reading arbitrary files or executing code on the machine running OpenRefine using a different set of import parameters in the database extension. This problem bears the vulnerability id CVE-2024-23833, reported by @l0n3rs.
  • (fixed since 3.7.5) A moderate vulnerabilities in the database extension was fixed. Connecting to a malicious MySQL server could read files or execute arbitrary code on the machine running OpenRefine. The vulnerabilities were assigned CVE-2023-41886 and CVE-2023-41887
  • identifiers respectively, and were reported by @nbxiglk0.
  • (fixed since 3.7.4) A moderate vulnerability in project import was fixed. Importing a maliciously crafted project could execute arbitrary code on the machine running OpenRefine. This vulnerability has been assigned the CVE-2023-37476 identifier. It was reported by Stefan Schiller from SonarSource.
  • New features:
  • Most text exposed to users in OpenRefine's UI can now be translated. Some strings (generated server-side) were not translatable so far. To help translators catch up on this backlog, do not hesitate to join us on Weblate. (#5030)
  • New media files can be uploaded to Wikibase instances such as Wikimedia Commons. The wikitext of existing files can also be edited thanks to the new fields introduced. (#4682)
  • A button "Discover Wikibase instances…" was added on the dialog which lists the registered Wikibase instances (#5007), whose design was improved (#5009)
  • In the Wikibase schema editor, statements with non-standard datatypes (such as EDTF dates or musical notations) are now supported, assuming they use strings as underlying representation (#3263)
  • The Wikibase issues tab now makes it possible to locate which rows are responsible for certain issues, using facets (#5033)
  • The default throttle delay for the "Add column by fetching URLs" operation was reduced to 500ms and the error reporting for this field was improved (#5188)
  • Wikibase templates (incomplete Wikibase schemas) can be saved and shared, as a way of helping contributors use the same way of structuring data in a Wikibase instance (#5043, #5303)
  • The line-based importer now supports a custom delimiter, instead of only newlines (#4103)
  • The Excel importer can be configured to import all cells as text, disabling the use of other datatypes supported by OpenRefine (#4838)
  • The "some value" and "no value" Wikibase values can now be uploaded by OpenRefine (#5360)
  • The Excel importer will also avoid coercing cell values to OpenRefine datatypes which do not fully fit them, such as representing a date as a date with time (#5389, #5390).
  • GREL changes:
  • Improved error handling in number formatting with the GREL toString function (#816)
  • The behaviour of the GREL function wholeText() has changed slightly in the way it handles newlines, following an upstream change in the jsoup library (jsoup issue #1636)
  • A new parent GREL function, to obtain the parent element of an XML element, was added (#5176)
  • Bug fixes:
  • The layout of the dialog to select a reconciliation match was improved so that the auto-complete widget does not hide the other options (#4821)
  • Better in-tool documentation around the way the scatterplot facet detects numerical columns (#4890)
  • The detection of URLs in cell values was fixed (#4546)
  • The error message displayed when trying to add a Wikibase manifest with a manifest version that is too old or recent was improved (#4847)
  • Errors returned by Jython expressions are more readable (#3012)
  • The ODS exporter no longer creates a default sheet "Sheet1" in the documents it creates (#4864)
  • Longer descriptions in auto-completion widget are not cut off anymore (#4988)
  • The interface for editing cell values was improved to better explain how to input dates (#3082)
  • The Windows refine.bat script was made more consistent with the Unix refine script (#4949, #5404)
  • The "Search for Match" dialog was rearranged so that the dropdown does not cover the buttons (#4945)
  • Error handling in the scatterplot facet was improved (#4893)
  • The "Collapse consecutive whitespaces" operation now handles unicode whitespace correctly (#4898)
  • (#4991)
  • The handling of GZIP-compressed files without .gz extension was improved in the importing pipeline (#547)
  • (#5153)
  • The "Add column based on this column dialog" can be submitted by pressing "Enter" in the column field (#5143)
  • The editing of redirected Wikibase items was fixed (#5162)
  • The user experience was improved in the case of incomplete Wikibase schemas (#5131)
  • The memory usage display was improved to show the used memory instead of the total memory, and was made more precise (#5222)
  • The association of labels to form inputs was improved, enhancing the accessibility of the interface (#5239, #5242, #5249, #5284)
  • An overflow issue in the reconciliation dialog was fixed (#5285)
  • The Wikibase manifests now properly support locally-running reconciliation endpoints (#5035)
  • The aspect ratio of Wikibase logos is now properly preserved (#5306)
  • The SQL exporter interface was improved (#5224)
  • The cell edit popup and dialogs with textbox inputs became resizable (#5330)
  • When marking a set of cells as "New" in an unreconciled column, the user is prompted for the reconciliation service to use (#4985)
  • More quality assurance checks were introduced in the Wikibase extension, such as checking for identical label and description in new Wikibase items (#4980)
  • The caching of auto-completion results in the Wikibase extension was fixed (#5190)
  • The Wikidata extension was fully renamed to "Wikibase extension" (#4525)
  • The controls of the cluster and edit dialog are greyed out while clustering is taking place (#5369)
  • The handling of unicode whitespace was improved throughout the application (#5105)
  • Our MacOS packages (.DMG) are now properly signed and notarized, which should make their installation easier (#4586). Also, the presentation of the DMG image was made more user-friendly by including the customary link to the Applications folder. (#5509)
  • The parsing of the unary minus sign in GREL was fixed (#5465)
  • (From 3.7-beta3 on) The clustering dialog no longer introduces non-breaking spaces when selecting options with spaces in them (#5581)
  • (From 3.7.1 on) The display of the memory usage during project import.

New in OpenRefine 3.7.7 (Nov 29, 2023)

  • A rendering issue in the Wikibase schema editor was fixed, by @wetneb (#6165)
  • Attempts to fetch invalid URLs in the "Add column by fetching URLs" operation are properly reported to the user, by @tfmorris (#6141)
  • A missing space between the "remove" and "configure" buttons of the "Add column from reconciled values" operation was added, by @SrinathKadam048 (#6151)

New in OpenRefine 3.7.6 (Oct 9, 2023)

  • A missing space was added in the layout of the text facet, by @frafra (#6071)
  • Browser launching on startup was fixed for Snap-packaged OpenRefine, by @kristbaum (#6065)
  • The selection of Wikibase statement merging strategies was fixed, by @wetneb (#6066)
  • A resizing issue in the presence of the Wikibase extension and facets was fixed, by @Abbe98 (#6070)

New in OpenRefine 3.7.5 (Sep 11, 2023)

  • A moderate vulnerability in the database extension was fixed. Connecting to a malicious MySQL server could read files or execute arbitrary code on the machine running OpenRefine. A CVE identifier for the vulnerability has been requested. The vulnerability was reported by @nbxiglk0.
  • New features
  • Most text exposed to users in OpenRefine's UI can now be translated. Some strings (generated server-side) were not translatable so far. To help translators catch up on this backlog, do not hesitate to join us on Weblate. (#5030)
  • New media files can be uploaded to Wikibase instances such as Wikimedia Commons. The wikitext of existing files can also be edited thanks to the new fields introduced. (#4682)
  • A button "Discover Wikibase instances…" was added on the dialog which lists the registered Wikibase instances (#5007), whose design was improved (#5009)
  • In the Wikibase schema editor, statements with non-standard datatypes (such as EDTF dates or musical notations) are now supported, assuming they use strings as underlying representation (#3263)
  • The Wikibase issues tab now makes it possible to locate which rows are responsible for certain issues, using facets (#5033)
  • The default throttle delay for the "Add column by fetching URLs" operation was reduced to 500ms and the error reporting for this field was improved (#5188)
  • Wikibase templates (incomplete Wikibase schemas) can be saved and shared, as a way of helping contributors use the same way of structuring data in a Wikibase instance (#5043, #5303)
  • The line-based importer now supports a custom delimiter, instead of only newlines (#4103)
  • The Excel importer can be configured to import all cells as text, disabling the use of other datatypes supported by OpenRefine (#4838)
  • The "some value" and "no value" Wikibase values can now be uploaded by OpenRefine (#5360)
  • The Excel importer will also avoid coercing cell values to OpenRefine datatypes which do not fully fit them, such as representing a date as a date with time (#5389, #5390).

New in OpenRefine 3.7.4 (Jul 17, 2023)

  • A moderate vulnerability in project import was fixed. Importing a maliciously crafted project could execute arbitrary code on the machine running OpenRefine. A CVE identifier for the vulnerability has been requested. The vulnerability was reported by @stefan-schiller-sonarsource.

New in OpenRefine 3.7.3 (Jun 26, 2023)

  • Starting openrefine.exe on Windows with Java 17 was fixed (#5583)
  • Wikibase edits on deleted items are skipped and do not stall the entire batch (#5385)
  • The HTML document language is aligned with the language of the interface (#5925)
  • The default reconciliation types are displayed with both name and id (#5907)
  • The transpose cells across columns was fixed so it treats blank cells as null (#5229)

New in OpenRefine 3.7.2 (Apr 6, 2023)

  • New since 3.7.1:
  • The German translation was fixed and other translations were updated (#5750)
  • New features:
  • Most text exposed to users in OpenRefine's UI can now be translated. Some strings (generated server-side) were not translatable so far. To help translators catch up on this backlog, do not hesitate to join us on Weblate. (#5030)
  • New media files can be uploaded to Wikibase instances such as Wikimedia Commons. The wikitext of existing files can also be edited thanks to the new fields introduced. (#4682)
  • A button "Discover Wikibase instances…" was added on the dialog which lists the registered Wikibase instances (#5007), whose design was improved (#5009)
  • In the Wikibase schema editor, statements with non-standard datatypes (such as EDTF dates or musical notations) are now supported, assuming they use strings as underlying representation (#3263)
  • The Wikibase issues tab now makes it possible to locate which rows are responsible for certain issues, using facets (#5033)
  • The default throttle delay for the "Add column by fetching URLs" operation was reduced to 500ms and the error reporting for this field was improved (#5188)
  • Wikibase templates (incomplete Wikibase schemas) can be saved and shared, as a way of helping contributors use the same way of structuring data in a Wikibase instance (#5043, #5303)
  • The line-based importer now supports a custom delimiter, instead of only newlines (#4103)
  • The Excel importer can be configured to import all cells as text, disabling the use of other datatypes supported by OpenRefine (#4838)
  • The "some value" and "no value" Wikibase values can now be uploaded by OpenRefine (#5360)
  • The Excel importer will also avoid coercing cell values to OpenRefine datatypes which do not fully fit them, such as representing a date as a date with time (#5389, #5390).
  • GREL changes:
  • Improved error handling in number formatting with the GREL toString function (#816)
  • The behaviour of the GREL function wholeText() has changed slightly in the way it handles newlines, following an upstream change in the jsoup library (jsoup issue #1636)
  • A new parent GREL function, to obtain the parent element of an XML element, was added (#5176)
  • For developers:
  • Some column metadata fields, which had never been exposed in the UI, have been removed (#5276)
  • A new extension point to customize cell rendering was introduced (#5154)

New in OpenRefine 3.7.1 (Mar 10, 2023)

  • New features:
  • Most text exposed to users in OpenRefine's UI can now be translated. Some strings (generated server-side) were not translatable so far. To help translators catch up on this backlog, do not hesitate to join us on Weblate.
  • New media files can be uploaded to Wikibase instances such as Wikimedia Commons. The wikitext of existing files can also be edited thanks to the new fields introduced.
  • A button "Discover Wikibase instances…" was added on the dialog which lists the registered Wikibase instances, whose design was improved
  • In the Wikibase schema editor, statements with non-standard datatypes (such as EDTF dates or musical notations) are now supported, assuming they use strings as underlying representation
  • The Wikibase issues tab now makes it possible to locate which rows are responsible for certain issues, using facets
  • The default throttle delay for the "Add column by fetching URLs" operation was reduced to 500ms and the error reporting for this field was improved
  • Wikibase templates (incomplete Wikibase schemas) can be saved and shared, as a way of helping contributors use the same way of structuring data in a Wikibase instance
  • The line-based importer now supports a custom delimiter, instead of only newlines
  • The Excel importer can be configured to import all cells as text, disabling the use of other datatypes supported by OpenRefine (#4838)
  • The "some value" and "no value" Wikibase values can now be uploaded by OpenRefine
  • The Excel importer will also avoid coercing cell values to OpenRefine datatypes which do not fully fit them, such as representing a date as a date with time
  • GREL changes:
  • Improved error handling in number formatting with the GREL toString function
  • The behaviour of the GREL function wholeText() has changed slightly in the way it handles newlines, following an upstream change in the jsoup library
  • A new parent GREL function, to obtain the parent element of an XML element, was added

New in OpenRefine 3.7.0 (Feb 15, 2023)

  • New features:
  • Most text exposed to users in OpenRefine's UI can now be translated. Some strings (generated server-side) were not translatable so far. To help translators catch up on this backlog, do not hesitate to join us on Weblate.
  • New media files can be uploaded to Wikibase instances such as Wikimedia Commons. The wikitext of existing files can also be edited thanks to the new fields introduced.
  • A button "Discover Wikibase instances…" was added on the dialog which lists the registered Wikibase instances, whose design was improved
  • In the Wikibase schema editor, statements with non-standard datatypes (such as EDTF dates or musical notations) are now supported, assuming they use strings as underlying representation
  • The Wikibase issues tab now makes it possible to locate which rows are responsible for certain issues, using facets
  • The default throttle delay for the "Add column by fetching URLs" operation was reduced to 500ms and the error reporting for this field was improved
  • Wikibase templates (incomplete Wikibase schemas) can be saved and shared, as a way of helping contributors use the same way of structuring data in a Wikibase instance
  • The line-based importer now supports a custom delimiter, instead of only newlines
  • The Excel importer can be configured to import all cells as text, disabling the use of other datatypes supported by OpenRefine
  • The "some value" and "no value" Wikibase values can now be uploaded by OpenRefine
  • The Excel importer will also avoid coercing cell values to OpenRefine datatypes which do not fully fit them, such as representing a date as a date with time
  • GREL changes:
  • Improved error handling in number formatting with the GREL toString function
  • The behaviour of the GREL function wholeText() has changed slightly in the way it handles newlines, following an upstream change in the jsoup library
  • A new parent GREL function, to obtain the parent element of an XML element, was added

New in OpenRefine 3.7 Beta 5 (Jan 28, 2023)

  • New features:
  • Most text exposed to users in OpenRefine's UI can now be translated. Some strings (generated server-side) were not translatable so far. To help translators catch up on this backlog, do not hesitate to join us on Weblate. (#5030)
  • New media files can be uploaded to Wikibase instances such as Wikimedia Commons. The wikitext of existing files can also be edited thanks to the new fields introduced. (#4682)
  • A button "Discover Wikibase instances…" was added on the dialog which lists the registered Wikibase instances (#5007), whose design was improved (#5009)
  • In the Wikibase schema editor, statements with non-standard datatypes (such as EDTF dates or musical notations) are now supported, assuming they use strings as underlying representation (#3263)
  • The Wikibase issues tab now makes it possible to locate which rows are responsible for certain issues, using facets (#5033)
  • The default throttle delay for the "Add column by fetching URLs" operation was reduced to 500ms and the error reporting for this field was improved (#5188)
  • Wikibase templates (incomplete Wikibase schemas) can be saved and shared, as a way of helping contributors use the same way of structuring data in a Wikibase instance (#5043, #5303)
  • The line-based importer now supports a custom delimiter, instead of only newlines (#4103)
  • The Excel importer can be configured to import all cells as text, disabling the use of other datatypes supported by OpenRefine (#4838)
  • The "some value" and "no value" Wikibase values can now be uploaded by OpenRefine (#5360)
  • The Excel importer will also avoid coercing cell values to OpenRefine datatypes which do not fully fit them, such as representing a date as a date with time (#5389, #5390).
  • GREL changes:
  • Improved error handling in number formatting with the GREL toString function (#816)
  • The behaviour of the GREL function wholeText() has changed slightly in the way it handles newlines, following an upstream change in the jsoup library (jsoup issue #1636)
  • A new parent GREL function, to obtain the parent element of an XML element, was added (#5176)
  • For developers:
  • Some column metadata fields, which had never been exposed in the UI, have been removed (#5276)
  • A new extension point to customize cell rendering was introduced (#5154)

New in OpenRefine 3.7 Beta 2 (Dec 12, 2022)

  • New features:
  • Most text exposed to users in OpenRefine's UI can now be translated. Some strings (generated server-side) were not translatable so far. To help translators catch up on this backlog, do not hesitate to join us on Weblate. (#5030)
  • New media files can be uploaded to Wikibase instances such as Wikimedia Commons. The wikitext of existing files can also be edited thanks to the new fields introduced. (#4682)
  • A button "Discover Wikibase instances…" was added on the dialog which lists the registered Wikibase instances (#5007), whose design was improved (#5009)
  • In the Wikibase schema editor, statements with non-standard datatypes (such as EDTF dates or musical notations) are now supported, assuming they use strings as underlying representation (#3263)
  • The Wikibase issues tab now makes it possible to locate which rows are responsible for certain issues, using facets (#5033)
  • The default throttle delay for the "Add column by fetching URLs" operation was reduced to 500ms and the error reporting for this field was improved (#5188)
  • Wikibase templates (incomplete Wikibase schemas) can be saved and shared, as a way of helping contributors use the same way of structuring data in a Wikibase instance (#5043, #5303)
  • The line-based importer now supports a custom delimiter, instead of only newlines (#4103)
  • The Excel importer can be configured to import all cells as text, disabling the use of other datatypes supported by OpenRefine (#4838)
  • The "some value" and "no value" Wikibase values can now be uploaded by OpenRefine (#5360)
  • The Excel importer will also avoid coercing cell values to OpenRefine datatypes which do not fully fit them, such as representing a date as a date with time (#5389, #5390).
  • GREL changes:
  • Improved error handling in number formatting with the GREL toString function (#816)
  • The behaviour of the GREL function wholeText() has changed slightly in the way it handles newlines, following an upstream change in the jsoup library (jsoup issue #1636)
  • A new parent GREL function, to obtain the parent element of an XML element, was added (#5176)
  • For developers:
  • Some column metadata fields, which had never been exposed in the UI, have been removed (#5276)
  • A new extension point to customize cell rendering was introduced (#5154)

New in OpenRefine 3.6.2 (Oct 3, 2022)

  • An overflow issue with the reconciliation dialog was fixed (#5286)

New in OpenRefine 3.6.1 (Aug 22, 2022)

  • New:
  • The editing of redirected Wikibase entities was fixed (#5162)
  • A bug with selection of clusters in the clustering dialog was fixed (#5138)
  • Date handling in the Google data extension was fixed (#5107)
  • A packaging issue in MacOS was tentatively fixed (#5160)

New in OpenRefine 3.6.0 (Jul 22, 2022)

  • New features:
  • The user is now warned when applying the "Fill down" or "Blank down" operations with a pending sorting criterion (#3256)
  • The import preview refreshing can be disabled (#4009)
  • Menu items to reveal collapsed columns were added to the column menus (#4067)
  • The path to the refine.ini configuration file can now be changed on the command line (#4113)
  • It is now possible to download the JSON representation of the operation history, without resorting to copy and paste (#4498)
  • It is now possible to work with Wikibase instances with federation enabled (#4287)
  • The merge strategy for statements can be configured in the Wikibase schema editor. This also adds support for deleting statements. Beware that schemas created with earlier versions of OpenRefine will still use the original merge strategy. (#3383, #2116, #4130)
  • OpenRefine can edit MediaInfo entities and not just Items (#4270)
  • It is possible to disable the new version notification by setting the configuration variable refine.display.new.version.notice=false (#4410)
  • The dialog to reorder and delete columns was improved to easily delete most columns (#4557)
  • The maximum editing speed and the Wikibase tag to apply to all Wikibase edits is now configurable for each Wikibase instance via its manifest (#3359)
  • Extra URL fields in the starting page can be removed thanks to a new button (#4606)
  • The "Use values as identifiers" operation now warns that it does not validate the identifiers (#3172)
  • GREL changes:
  • A new GREL function, parent, was introduced to obtain the parent element of an XML element (#4181)
  • A new GREL function, scriptText, was introduced to obtain the text contained in a <script> or <style> element in HTML (#4189)
  • The random (previously randomNumber) GREL function was improved (#3143)
  • A new GREL function parseUri was introduced (#1857)
  • A new GREL function detectLanguage was introduced (#642)
  • New GREL functions encode and decode were introduced (#148)
  • The error handling of the pow and exp functions was improved (#3062)
  • The division operator returns NaN when computing 0/0 (#377)
  • A function timeSinceUnixEpochToDate was introduced, to convert a duration since Epoch to a date object (#608)
  • A function replaceEach was introduced, to replace multiple substrings in one go (#2606)

New in OpenRefine 3.6 RC 1 (Jul 5, 2022)

  • New features:
  • The user is now warned when applying the "Fill down" or "Blank down" operations with a pending sorting criterion (#3256)
  • The import preview refreshing can be disabled (#4009)
  • Menu items to reveal collapsed columns were added to the column menus (#4067)
  • The path to the refine.ini configuration file can now be changed on the command line (#4113)
  • It is now possible to download the JSON representation of the operation history, without resorting to copy and paste (#4498)
  • It is now possible to work with Wikibase instances with federation enabled (#4287)
  • The merge strategy for statements can be configured in the Wikibase schema editor. This also adds support for deleting statements. Beware that schemas created with earlier versions of OpenRefine will still use the original merge strategy. (#3383, #2116, #4130)
  • OpenRefine can edit MediaInfo entities and not just Items (#4270)
  • It is possible to disable the new version notification by setting the configuration variable refine.display.new.version.notice=false (#4410)
  • The dialog to reorder and delete columns was improved to easily delete most columns (#4557)
  • The maximum editing speed and the Wikibase tag to apply to all Wikibase edits is now configurable for each Wikibase instance via its manifest (#3359)
  • Extra URL fields in the starting page can be removed thanks to a new button (#4606)
  • The "Use values as identifiers" operation now warns that it does not validate the identifiers (#3172)
  • GREL changes:
  • A new GREL function, parent, was introduced to obtain the parent element of an XML element (#4181)
  • A new GREL function, scriptText, was introduced to obtain the text contained in a <script> or <style> element in HTML (#4189)
  • The random (previously randomNumber) GREL function was improved (#3143)
  • A new GREL function parseUri was introduced (#1857)
  • A new GREL function detectLanguage was introduced (#642)
  • New GREL functions encode and decode were introduced (#148)
  • The error handling of the pow and exp functions was improved (#3062)
  • The division operator returns NaN when computing 0/0 (#377)
  • A function timeSinceUnixEpochToDate was introduced, to convert a duration since Epoch to a date object (#608)
  • A function replaceEach was introduced, to replace multiple substrings in one go (#2606)
  • For extension developers:
  • (from 3.6-rc1 on) We migrated to jQuery 3.6.0. If you are using jQuery in your extension, some jQuery syntaxes that have been deprecated earlier might have been removed. If your extension runs with OpenRefine 3.5.2, you can check the web developer console for warning messages when the extension is used: fixing those should be enough for your extension to be compatible with OpenRefine 3.6 (#4891)

New in OpenRefine 3.6 Beta 2 (Jun 7, 2022)

  • New features:
  • The user is now warned when applying the "Fill down" or "Blank down" operations with a pending sorting criterion (#3256)
  • The import preview refreshing can be disabled (#4009)
  • Menu items to reveal collapsed columns were added to the column menus (#4067)
  • The path to the refine.ini configuration file can now be changed on the command line (#4113)
  • It is now possible to download the JSON representation of the operation history, without resorting to copy and paste (#4498)
  • It is now possible to work with Wikibase instances with federation enabled (#4287)
  • The merge strategy for statements can be configured in the Wikibase schema editor. This also adds support for deleting statements. Beware that schemas created with earlier versions of OpenRefine will still use the original merge strategy. (#3383, #2116, #4130)
  • OpenRefine can edit MediaInfo entities and not just Items (#4270)
  • It is possible to disable the new version notification by setting the configuration variable refine.display.new.version.notice=false (#4410)
  • The dialog to reorder and delete columns was improved to easily delete most columns (#4557)
  • The maximum editing speed and the Wikibase tag to apply to all Wikibase edits is now configurable for each Wikibase instance via its manifest (#3359)
  • Extra URL fields in the starting page can be removed thanks to a new button (#4606)
  • GREL changes:
  • A new GREL function, parent, was introduced to obtain the parent element of an XML element (#4181)
  • A new GREL function, scriptText, was introduced to obtain the text contained in a <script> or <style> element in HTML (#4189)
  • The random (previously randomNumber) GREL function was improved (#3143)
  • A new GREL function parseUri was introduced (#1857)
  • A new GREL function detectLanguage was introduced (#642)
  • New GREL functions encode and decode were introduced (#148)
  • The error handling of the pow and exp functions was improved (#3062)
  • The division operator returns NaN when computing 0/0 (#377)
  • A function timeSinceUnixEpochToDate was introduced, to convert a duration since Epoch to a date object (#608)
  • A function replaceEach was introduced, to replace multiple substrings in one go (#2606)

New in OpenRefine 3.5.2 (Jan 26, 2022)

  • Log4j was upgraded to 2.17.1

New in OpenRefine 3.5.1 (Dec 19, 2021)

  • Log4j was upgraded to version 2.16.0
  • OpenRefine is compatible with Java versions 8 to 17 (#4106)

New in OpenRefine 3.5.0 (Nov 7, 2021)

  • New features:
  • Wikidata support has been generalized to arbitrary Wikibase instances. (#1640)
  • The cross function now accepts implicit project and column names (#2504)
  • The left panel can be collapsed (#1038)
  • Support for more Wikidata constraints was added (multi-value, difference within range, conflicts with, and citation needed constraints) (#2354)
  • Splitting multi-valued cells is now possible by transition between uppercase/lowercase (#2238)
  • When importing multiple archive files, importers can store the filename of the archive file each row was extracted from (#1963)
  • It is now possible to go to a page of the project table directly (#2638)
  • The pagination sizes offered by the UI can now be configured by setting the ui.browsing.pageSize preference to values such as [100,500,1000,2000] (#2624)
  • Format detection at the import stage was improved (#2805, #2800)
  • The split/join multivalued cells dialogs now remember the last separator used (#2197)
  • The forEach GREL function works on JSON objects (#3149)
  • A new GREL function wholeText can be used to extract all the text inside an XML element (including in its children) (#3180)
  • A dialog to confirm the removal of starred expressions was added (#501)
  • HTTP host validation was added (#3288)
  • The Wikibase extension can now be used to add BCE dates (#3816)
  • The common cell transforms can be run on a selection of columns easily (#1843)
  • Greater numbers of rows per page can now be selected (#3249) (after 3.5-beta1)

New in OpenRefine 3.5 Beta 1 (May 30, 2021)

  • New features:
  • Wikidata support has been generalized to arbitrary Wikibase instances. (#1640)
  • The cross function now accepts implicit project and column names (#2504)
  • The left panel can be collapsed (#1038)
  • Support for more Wikidata constraints was added (multi-value, difference within range, conflicts with, and citation needed constraints) (#2354)
  • Splitting multi-valued cells is now possible by transition between uppercase/lowercase (#2238)
  • When importing multiple archive files, importers can store the filename of the archive file each row was extracted from (#1963)
  • It is now possible to go to a page of the project table directly (#2638)
  • The pagination sizes offered by the UI can now be configured by setting the ui.browsing.pageSize preference to values such as [100,500,1000,2000] (#2624)
  • Format detection at the import stage was improved (#2805, #2800)
  • The split/join multivalued cells dialogs now remember the last separator used (#2197)
  • The forEach GREL function works on JSON objects (#3149)
  • A new GREL function wholeText can be used to extract all the text inside an XML element (including in its children) (#3180)
  • A dialog to confirm the removal of starred expressions was added (#501)
  • HTTP host validation was added (#3288)
  • The Wikibase extension can now be used to add BCE dates (#3816)
  • The common cell transforms can be run on a selection of columns easily (#1843)

New in OpenRefine 3.4.1 (Sep 24, 2020)

  • The Google Drive and Google Sheets integration was fixed.

New in OpenRefine 3.4 Beta (May 14, 2020)

  • We now offer a Windows package with embedded Java runtime engine (no need to install Java with this one) (#2272)
  • Importing data from SQLite is now possible (#1951)
  • Modal dialogs can now be closed with the ESC key (#1018)
  • A cell.error field has been added, to fetch the error message stored in a cell (#525)
  • A new menu item was created to extract entity identifiers from a reconciled column (#1975)
  • It is now possible to quote all cell values in the custom tabular exporter (#1869)
  • An option was added in the CSV/TSV importer to strip whitespace in cell values (#791)
  • The Google Sheets and Google Drive export have been added to the main "Export" menu (#2453)
  • The cross function is no longer restricted to source values from the column where it is invoked (#1950)
  • The cross function now works for any type of cell value (#2461)
  • It is now possible to configure the maxlag value used by the Wikidata extension by setting wikibase:upload:maxLag to some integer in the preferences (#2304)
  • Facets can be minimized (#2553)

New in OpenRefine 3.3 Beta (Oct 21, 2019)

  • New features:
  • A new menu for joining (concatenating) columns has been added (#2109)
  • Commonly used fields in dialogs now have autofocus (#2130)
  • The Wikidata extension now supports adding dates with custom calendars (#2136)
  • Vulnerabilities:
  • A cross-site scripting vulnerability in the database extension was fixed (#2151)
  • Cross-Site Request Forgery (CSRF) protection was added to POST API endpoints. If you rely on OpenRefine's server API you will need to adapt your calls accordingly (#2164)

New in OpenRefine 3.2 Beta (Mar 1, 2019)

  • New features:
  • New action to replace smart quotes to their ASCII equivalent (#1676)
  • New phonetic clustering methods are available: Beider-Morse (#926) and Daitch-Mokotoff (#927)
  • The "Uses values as identifiers" operation now accepts cells with RDF uris instead of just identifiers, using the identifierSpace declared by the reconciliation service (#1953)
  • References in the Wikidata schema can be copied across statements and items (#1912)
  • Items suggested by auto-complete can now be clicked with the middle button, which opens their URL in a new tab (#1934)
  • Reconciliation previews are now shown when hovering the candidate (no click is needed). Clicking the candidate opens its page in a new tab. It is possible to disable this feature for matched cells by adding cell-ui.previewMatchedCells=false in the preferences. (#1943)

New in OpenRefine 3.1 Beta (Nov 8, 2018)

  • New features:
  • Importing n-triples, ttl, and JSON-LD files is now possible (#1758)
  • The smartSplit function now supports any string, not just a single character (#1761)
  • A new menu to search and replace was added (#1742)
  • A field to specify custom column names was added in the CSV/TSV importer
  • It is now possible to import and export a Wikidata schema in JSON (#1776)
  • Strings are now automatically trimmed in Wikidata schemas. The corresponding issues have been removed. (#1781)
  • Browser-based autocomplete has been enabled for Wikidata edit summaries. (#1596)
  • It is now possible to mark a column of identifiers as reconciled without calling the reconciliation service. (#1778)

New in OpenRefine 3.0 Beta (May 28, 2018)

  • New features:
  • Wikidata extension
  • Data package metadata
  • Tag system
  • Google drive API
  • OpenRefine Database Import Extension
  • Add coalesce function
  • Implement "Facet by null" and "Facet by empty string" and add to customized facets menu
  • Feature Request: Export SqlDump
  • Migrate from JRDF to JENA library
  • Added option to toggle show/hide null values in cells in data-table
  • Unify the internal date type
  • Update OpenRefine logo
  • Set http req headers
  • Add find function
  • Some bug fixes

New in OpenRefine 2.8 (Nov 20, 2017)

  • New features:
  • Project metadata support
  • Enhancement of the reconciliation API
  • Support split multivalued-cells by regex/special characters
  • Text filter exclude
  • Add free memory detection and Notification to user
  • Improved UI for better usability
  • New importer for Wikitables
  • Some bug fixes

New in OpenRefine 2.7 (Nov 20, 2017)

  • New features:
  • Wikidata Reconcile (replaced old Freebase Reconcile service) and hosted by Wikimedia Foundation.
  • Export Clusters button on Clustering dialog.
  • Japanese translation
  • Support multiple "logical and" and "logical or" instead of just 2
  • "Transform All" support to apply the operations to multiple columns
  • Some bug fixes

New in OpenRefine 2.6 RC 2 (Oct 13, 2016)

  • NOTE: Release candidate for OpenRefine v2.6. Hopefully no major changes are required except the final versioning and some minor improvoment. Please file issues if found any! USE AT YOUR OWN RISK

New in OpenRefine 2.6 RC 1 (Oct 13, 2016)

  • NOTE: This isn't a real release candidate at this point, but rather a trial build for the development team to check out how many loose ends are left to clean up. Please continue to use the v2.6 beta 1 release for now. USE AT YOUR OWN RISK

New in OpenRefine 2.6 Beta.1 (Apr 14, 2014)

  • Features:
  • Issue 144: Use subdirectory for archives (tar, zip)
  • Issue 467: Provide feedback to user on JVM heap memory usage
  • Issue 524: Shorten name of anonymous JSON nodes in column names
  • Bugs fixed:
  • Issue 154: Can't import RDF/XML Data
  • Issue 226: Load into freebase: date objects are not converted into proper freebase dates
  • Issue 361: Headless operation on OS X leaves Dock icon bouncing
  • Issue 390: Latest POI fixes some import issues
  • Issue 432: cross() failing
  • Issue 436: Google refine don't start on debian squeeze
  • Issue 490: Unable to guess filetype is CSV when importing a simple CSV file
  • Issue 496: google-refine-2.5-rc2-r2379 failed to start if using refine.ini
  • Issue 515: "Open Project" page shows negative interval for "Last Modified"
  • Issue 517: Combin() function does not work as intended
  • Issue 521: Machine readable string (/type/rawstring) missing
  • Issue 523: Return all available HTTP error information on when Fetch URL fails
  • Issue 535: Refine fails to import Excel 2010 XLSX file with null hyperlinks
  • Issue 537: ToNumber should try to convert to integer first
  • Issue 541: Can't constrain a Freebase key query when using add column
  • Issue 542: ToDate should work on integers
  • Issue 543: Project creation from URL can't handle gzip Content-Encoding
  • Issue 544: FileNotFound exception when importing archive from URL
  • Issue 548: escape(value, encoding) returns "null" for non-string values
  • Issue 551: MQL preview missing "insert" clause
  • Issue 553: Upload Error - Not a directory
  • Issue 554: Format guesser ranking isn't working correctly in some cases
  • Issue 558: 'refine windows_dist' does not work on MacOS Lion
  • Issue 559: Deadlock between autosave thread and history code
  • Issue 578: json import on create project deletes useful blank data from arrays
  • Issue 586: Only one parse date format is attempted from list in toDate(format1,format2)
  • Issue 594: Date diff function doesn't work for two Calendar objects
  • Issue 596: JSON importer should not deserialize nulls as the string "null"
  • Issue 597: Deselect all button on Custom Tabular Exporter dialog doesn't work
  • Issue 599: RDF/XML parser preview not wired up
  • Issue 601: Allow selection of root element on JSON import
  • Issue 604: The common transform “Trim leading and trailing whitespace” doesn’t trim non-breaking spaces