Softpedia >Windows >Internet >Other Internet Related >FMiner Professional >  Changelog

FMiner Professional Changelog

What's new in FMiner Professional 9.81

Jul 25, 2015
  • Added path of ftp report, http://www.fminer.com/run-settingsclear-tables-run-traverse-table-rows-report-after-ru/
  • Fixed bug of "Prompt input a row to table".
  • Fixed the issue: when select big page's content, program will be very slow.

New in FMiner Professional 9.80 (Jul 15, 2015)

  • Added the feature:
  • Upload resutls to ftp.
  • Create screenshot of page

New in FMiner Professional 9.79 (May 19, 2015)

  • Fixed the bug: For "openlinks" action, when "Links type" is "Generate URLs", the UI can't update when switch to other actions.
  • Fixed the bug: "open recent project" not work for Mac OS edition.
  • For "capture content" action, add
  • Added loading tip when the page not finish loading.

New in FMiner Professional 9.78 (Feb 28, 2015)

  • Fixed the bug: when here's some data in a table, add new column, the table can't be exported.
  • Change location of activation inforamtion of Mac Edition to system, then update to new version, no need activate it again.

New in FMiner Professional 9.77 (Feb 2, 2015)

  • Fixed the bug: downloading file not work, this is a new bug in 9.76.
  • Fixed the bug: error somethings when call table.select_rows in run code.
  • Fixed the bug: missed newlines when scraped target is hidden. (e.g. the hidden table).
  • Added verification mechanism to prevent special characters in "table name" and "column name".

New in FMiner Professional 9.76 (Jan 26, 2015)

  • Add Downloader, this is a tool to download files of URLs in a data table.
  • Add a pager to data tables, can use it go to "first", "previous", "next", "last" page to check all data in data table.

New in FMiner Professional 9.75 (Jan 14, 2015)

  • Fixed bug: when stop manually once, "run multiple times with each row data of a table" will not work, it will just travel one row and stop until close and open the project again.
  • Enabled "LocalStorageEnabled" in webkit settings, encountered a html5 site can't work without LocalStorageEnabled, enable it to make the site work.
  • Added "import_xls" and "import_csv" to table object of "run code", and added a template of "run code" to import xls/csv.
  • Changed "remove data", now click the button will remove all selected rows(The old version just remove the first row).

New in FMiner Professional 9.74 (Dec 22, 2014)

  • Fixed a serious bug: The execution will become slower and slower when run a long time. This bug does not effect much for the short time extraction.
  • Add escape character for proxy.

New in FMiner Professional 9.73 (Dec 9, 2014)

  • Fixed bugs:
  • when "new project", settings not update.
  • "click" action can't work for a site.
  • New features:
  • Import/export tables structure, it's important for make many projects with the same data structure.
  • Add "Generate URLs" to "openlink(s)" action

New in FMiner Professional 9.72 (Oct 31, 2014)

  • Fixed the bug: Captcha get/input can't work.

New in FMiner Professional 9.71 (Oct 20, 2014)

  • Fixed the problem: projects on windows or Mac OS can't be opened on different system with the error “can't find module copy_reg” sometimes. Now changed project format to binary format.

New in FMiner Professional 9.70 (Oct 11, 2014)

  • Upgrade the core of QT to last version 4.8.6, it include new version of webkit, so can open some sites can't be open previously, such as instagram.com. http://www.fminer.com/forum/topic/248/
  • Changed the default folder of downloaded files to path of project.
  • Change the log window just show last 100 logs, this will save the resource for long run project.
  • Add "with class ..." in menu of "Change Xpath"
  • Fixed the problem of "parent URL".
  • Remove limit of 50 fields in table, now you can add as many fields.
  • Fixed the "rich text" format error of the "adjust data".
  • Made the scene size bigger of complex project.
  • Fixed bugs: can't capture "description" and "keywords" in meta when they are upper case.

New in FMiner Professional 9.63 (Jun 27, 2014)

  • 9.62 didn't fix all bugs of " ", release this version to fix the issue.

New in FMiner Professional 9.62 (Jun 27, 2014)

  • Fixed bug in 9.6, select table show all blank to " "

New in FMiner Professional 9.61 (Jun 27, 2014)

  • Fixed the bug in 9.60, can't show images on pages.

New in FMiner Professional 9.60 (Jun 27, 2014)

  • Fixed some bugs when designing the project with iframe on pages.
  • Fixed the bug, when select big images on page, the program very slow to freeze.
  • Add a feature "Prompt input a row to table" before run.
  • Add export results to "xml/html" and "json" format.

New in FMiner Professional 9.50 (Apr 25, 2014)

  • Changed "regular expression" capture. Add a helper dialog to capture content with "regular expression".
  • To stop the spam on forum, add a menu "register forum account" to FMiner. Disabled register account on site.
  • Fixed some bugs.

New in FMiner Professional 9.30 (Apr 14, 2014)

  • This version fixed the bug:
  • Can't set "max recursive level" on "openlink(s)" action.
  • Changes:
  • Change the lines smooth.
  • For selected DOM on pages, now will change color of backgroup, not just the border, because some borders of tr in tables often not show.
  • Add numbers for multiple targets on "selection" panel.

New in FMiner Professional 9.20 (Apr 14, 2014)

  • Fixed bug can't scrape text from "select" on pages.
  • Add feature can "down", "up" the fields in "data table".
  • Add "referer" in header of request when download file from pages.
  • Add "useragent" control on "settings dialog, now can change it here not "useragent.ini" file.
  • Add "clear table" template in "run code".
  • Changed "Batch add URLs", now program will record old "plattern" and keywords.

New in FMiner Professional 9.11 (Apr 14, 2014)

  • Some small changes on UI.
  • Fixed the bug: when remove link on scene, program crash sometimes.
  • Fixed the bug: Argument '--expert_headers' can't work.
  • Added new argument '--move_errorlinks' for execute task externally.

New in FMiner Professional 9.03 (Apr 14, 2014)

  • Fixed bug of "wait download" http://www.fminer.com/capture-content/.

New in FMiner Professional 9.02 (Apr 14, 2014)

  • Fixed bug of "Run multiple times with each row data of a table".
  • Add missing templates of "run code" for pro version.
  • Changed http://www.fminer.com/execute-fminer-task-externally/ , now FMiner can just export data without run(resume) options.

New in FMiner Professional 9.01 (Apr 14, 2014)

  • Fixed the bug of "capture" action, "DOM attribute" type can't scrape some attribute such as tag, innterText. This bug just in version 9.00.

New in FMiner Professional 9.00 (Apr 14, 2014)

  • Update SDK of PySide from 1.11 to 1.21, the new version of PySide included the new version of webkit, it's more stable.
  • Add feature: Added options for the "javascript dialog" on settings dialog.
  • Fixed bug: Browsers' loading icons not show.

New in FMiner Professional 8.24 (Sep 21, 2013)

  • Fixed bug: when step run "scrape action", not update data table for new data
  • Add feature: when run a project, will created a project.running.pid file, and when done the extraction, will created a project.done.pid file. Then you can know the program status and pid.

New in FMiner Professional 8.23 (Sep 18, 2013)

  • Fixed bug, when add the first field to table, will be repeated twice.
  • Changed select action, now can select options with index format like "#1".
  • Changed position of XPath, now can input data here.

New in FMiner Professional 8.22 (Sep 16, 2013)

  • Add a feature "Insert Data to Column", with this feature, you can insert many string or number to a column of a table.
  • Fixed bug: can't really stop the program with option of "Run multiple times with each row data of a table".

New in FMiner Professional 8.20 (Sep 16, 2013)

  • Fixed a bug in core of "adblock" and "block files".
  • Add new feature to filter of "open links" action, now it can allow/ban URLs in a column of table. This feature can be used for "extract Incrementally", for example, you can capture the URL of a page to a column when save data, then set the filter to ban URLs in this column, the program will not open these links scraped when run again.
  • Add a feature "Get License Number from Invoice Number", because some users can't get the email with the license number when order, the email often be discarded by spam filter. With this feature, you can get license number directly if you can't get the email.

New in FMiner Professional 8.15 (Sep 16, 2013)

  • Fixed bug: often "Trial expires" for the new user.

New in FMiner Professional 8.14 (Sep 16, 2013)

  • Fixed bug: 'runto' can't work.

New in FMiner Professional 8.12 (Aug 23, 2013)

  • 8.11 did not really fix the bug in activating, this version fixes it.

New in FMiner Professional 8.11 (Aug 23, 2013)

  • Fixed the bug: get the error message "None type object has no attribute 'strip'" sometimes when active the program.

New in FMiner Professional 8.10 (Aug 15, 2013)

  • This version fixed bug in "Automatic upgrade" of version 8.00, 8.00 can't update automatically, so if you are using 8.00, you should download and install 8.10 manually.
  • Add a feature of "Run multiples times with each row data of a table" for Pro version, with this feature you can input more than one sets of data to form of page easily, does not need such a complex operation http://www.fminer.com/input-every-row-data-table-1/.

New in FMiner Professional 8.00 (Aug 8, 2013)

  • Fixed bug of "scheduler".
  • Add new action "scroll down page".
  • Add "Add a unique time suffix to the output file/folder name" on export config dialog, you can export data to a different file with this option.
  • Add some arguments for executing Fminer task externally.

New in FMiner Professional 7.05 (May 7, 2013)

  • Fixed the bug: when add a column to data table, it not been recorded in project file(.fmpx), just in database file(*.db), then when remove and rebuild the database file, the columns will miss.

New in FMiner Professional 7.04 (Apr 10, 2013)

  • Add a feature "Get data from file" see: http://www.fminer.com/input-data/
  • Fixed the bug: program will crash sometimes when remove node from the scene.

New in FMiner Professional 7.03 (Apr 10, 2013)

  • Add "adjust data with javascript" in "caputure" action, then you can ajust the scraped data before saving it.
  • Add "log" action, it can be used to control the flow, and record error links, and report failed captcha to third-party captcha decoding services to save money.

New in FMiner Professional 7.01 (Apr 10, 2013)

  • Fixed the bug "URLs filter" in "openlinks" action can't work.
  • Fixed the bug "Max recursive level" not work correctly.

New in FMiner Professional 7.00 (Apr 10, 2013)

  • Change project file format, seperate the project file from the database, so It can add more database format as the internal database in later version.
  • "Validate" action add "js result" option, you can write some js code here, and it will validate the returned value.
  • Add "open link with image" option to the "change target xpath", it's for locate the image link without text.
  • Add "not wait when done" to the actions, when check it, when done the action, FMiner will not wait to do the next action.
  • Fixed some bugs.

New in FMiner Professional 6.62 (Apr 10, 2013)

  • Add "Save code and screenshot of waring/error pages" option on settings dialog, you can uncheck it for running the big project to get better performance.
  • Add "Run scripts to edit data" for edit the scraped data.

New in FMiner Professional 6.61 (Feb 18, 2013)

  • Fixed bug when export xlsx

New in FMiner Professional 6.58 (Jan 31, 2013)

  • Fixed a bug to export xlsx file.

New in FMiner Professional 6.57 (Jan 30, 2013)

  • Add support for exporting and importing xlsx(Excel 2007) file.

New in FMiner Professional 6.56 (Jan 28, 2013)

  • Fixed the bug: Click "run to current action", FMiner not stop when get to the selected action.

New in FMiner Professional 6.55 (Jan 24, 2013)

  • Remove "page back", "page forward", "page refresh", because they can be realized by with javascript in "run javascript code" action.
  • Add "Clear up cookies footprint".

New in FMiner Professional 6.53 (Jan 21, 2013)

  • Remove "set and change proxy" in "before do action", because it's conflict with the "rotate proxy", and not very useful.
  • Remove "Do follow actions on initial page" option in "open link(s)" action, because most sites need enable it when "Open link(s) recursively" enabled, and It is difficult to understand. Now when "Open link(s) recursively" enabled, it will be enabled implicitly.
  • Make the "Pages" number when running correctly.

New in FMiner Professional 6.51 (Jan 17, 2013)

  • Add a "Block files" option, then can enable it to block all additional files of pages, eg. js, css, jpg... files.

New in FMiner Professional 6.50 (Jan 14, 2013)

  • Add a feature of "rotate proxies every some pages/minutes", then it can use list of proxies easily.
  • Change format of "scraped text", now the scraped text will like the copy from page to notepad with delimiter.

New in FMiner Professional 6.49 (Jan 7, 2013)

  • This version fixed the bug: adblcok often crash the program, and reduced memory usage.

New in FMiner Professional 6.47 (Jan 3, 2013)

  • This version add a "relatively select" button for locate targets on pages, change the default XPath for selected targets in "capture content" node. They are all for positioning targets accurately for capture contents from bulk pages.
  • Make some optimization for running project: 1.disable updating data tables when running. 2. disable scene and attribute panel updating when running. 3. disable "web inspector" of the main browser when running.

New in FMiner Professional 6.46 (Jan 3, 2013)

  • This version add "parent url" in "capture content" nodes, then we can scrape the parent page's url, it's helpful to associate multiple tables hold the data scraped from different pages. For example, you can capture url and other data to table 1 from a page, then open links from this page and save scrape data and "parent url" from these children pages to table 2, then the "url" in table 1 will be same with the "parent url" in table 2

New in FMiner Professional 6.45 (Dec 27, 2012)

  • This version fixed a problem when "goto" and "open link(s)" have multiple links. Now the new opened browsers will have the same global variables for the "goto" and "open link(s)" nodes.

New in FMiner Professional 6.44 (Dec 27, 2012)

  • Add "write results to variables" option in "scrape page" nodes. It can be used for writing script codes in "runcode" nodes.

New in FMiner Professional 6.43 (Dec 17, 2012)

  • Limit the details of blocks on the selection widget to top 10 for performance. This will avoid pages freeze when the pages are big and many targets are selected.
  • Add static data type in "capture content" node, and you can save some static data to the tables with it.
  • Make the cursor busy when page loading to avoid misuse.
  • Add copy data name in context menus of data tables and variables, this is to facilitate to write scripts.
  • Fixed bug: when more than one scrape page nodes with multiple data tables, the assigned tables often change incorrectly when switch between these scrape page nodes.
  • Fixed bug: when a Goto node is not the starting node, and has bulk links, only the first link be valid

New in FMiner Professional 6.42 (Nov 28, 2012)

  • This version changed welcome dialog, and add some links on it to help the new users.
  • Fixed the bug: When add "capture content[download]" by right clicking on the page during record, the target is not the link sometimes.

New in FMiner Professional 6.41 (Nov 22, 2012)

  • When add a recursive "open link(s)" node, "Do follow actions on initial page" will be checked by default.

New in FMiner Professional 6.40 (Nov 8, 2012)

  • Fixed bug: when the page has more than one frames, can't select the target(s) on it. It's a major bug, and cause tutorial 3 and tutorial 4 can't work correctly, and I launched the new version so quickly is because of this bug.
  • Add max recursive level in recursive open link(s) node, then you can set the deep of the spider can go.
  • Add URL filter in open link(s) node, then you can select bulk of links and set a filter just allow the special links.
  • Add regular express option in capture content node, it can extract the data from the target's html code, you can use it to extract some special data(eg, email address).

New in FMiner Professional 6.32 (Nov 6, 2012)

  • Add Non-empty attribute to data field, then if it can't scrape content for this field, FMiner will ignore and doesn't save this data.
  • Add a default folder for hold the downloaded files when a capturing node is downloading type, default folder is "files".
  • Add an option "capture content [download]" to the menu when record actions, it's for downloading image or files.
  • Fixed bug: When select a "capture content" node, run to will never reach it.

New in FMiner Professional 6.31 (Nov 5, 2012)

  • Change the model of "select target" and "group select", now, when clicked button of "select target" or "group select", after click the target on page to select it, the checked button will raise automatically, so you can continue to do other things(eg, record actions). This is convenient for recording macro, and reduced the operating steps. Because it's a big change for operation, and some video tutorials need change.
  • Change the model of tip window, we have changed tip contents shorter and the tip widget will popup at every steps, and add links on them to open the document of the step or current node(action).
  • Add timestamp at the end of each line of the logs.
  • Add feature of doing "compact database" when "clear data table", because without "compact database", the project will not reduce and will become bigger and bigger when add and clear data frequently.
  • Fixed bug: When clicked a link on log window, later added logs will be abnormal.
  • Fixed bug: The result URLs of "open link(s)" nodes are not correct and just show text not links at some times.
  • Fixed bug: When click "clear" button to clear all nodes, the attribute panel doesn't update.

New in FMiner Professional 6.23 (Oct 16, 2012)

  • Fixed the bug: when change focus from a "validate" node to other node, target panel disappear sometimes.
  • Add a "backup the project to..." option in file menu for backup the project.
  • Now FMiner will record "page screenshot", "page code", and "page url" in logs, when action fail or scrape nothing, and this can facilitate debugging.

New in FMiner Professional 6.22 (Oct 15, 2012)

  • Add an option of “Do follow actions on initial page” in recursive node, then all following actions can be done for the initial page before click “next” link.

New in FMiner Professional 6.21 (Oct 8, 2012)

  • Add Try scrape once when no blocks found option on settings dialog.
  • Add transfer license to another computer feature

New in FMiner Professional 6.20 (Oct 1, 2012)

  • Fixed the bug when export tables to database.
  • Add support of sqlite export.
  • Add "open link(s) recursively" option in open link(s) actions. This is an important improvement, now no need drag and drop line to open "next link" recursively. (Need some new tutorials to replace the old to show this feature)

New in FMiner Professional 6.17 (Oct 1, 2012)

  • This version add a feature of automatic upgrade. So you can keep using the last version without reinstalling the entire program frequently.

New in FMiner Professional 6.15 (Oct 1, 2012)

  • More choices in “change xpath” menu, Added some “short path” and “starts with” for targets, so you can try these when program can't find target.
  • Add an “export column headers” option on “export config” dialog, when it's checked, the first row of exported data(csv or xls file) will be the names of columns.
  • Add “batch add urls” features, can be used to add bulk links to the “goto” action.

New in FMiner Professional 6.14 (Oct 1, 2012)

  • Add a combo box for scaling the scene.
  • Change the proxy format from “ip:port|type” to “ip:port@type”, this is for compatibility with some proxy software. e.g proxyfire.
  • Changed the default useragent of the browser, now you can change useragent in useragent.ini file.
  • Fixed the bug when link “when fail” joint to a “wait” action, can't work correctly.
  • Add a menu to all diagram joints for adding “action” from the joint.

New in FMiner Professional 6.13 (Oct 1, 2012)

  • It fixed some bugs, and made the javascript error message to the file “fminer.log” and not show in the log window to the log window clearer.

New in FMiner Professional 4.01 (Dec 28, 2011)

  • Bug fixed:
  • Advanced button on macro attribute panel can't work.
  • Fixed bug on button Select content out of block.
  • Small improvement:
  • Add a button 'disable selection temporarily', and 'Open custom url' in url combobox. They are for testing the selection on a page.
  • Add a label of 'Done action' to display the actions finished.

New in FMiner Professional 4.00 (Dec 21, 2011)

  • We have added two new attributes for every action in a macro node:
  • Use data pattern
  • Targets act one by one
  • For a looping node(navigation or macro), we add Max loop count in its Advanced settings

New in FMiner Professional 3.00 (Dec 21, 2011)

  • We have added two new attributes for every action in a macro node:
  • Use data pattern
  • Targets act one by one
  • For a looping node(navigation or macro), we add Max loop count in its Advanced settings

New in FMiner Professional 2.10 (Dec 21, 2011)

  • This release adds a wizard dialog box, which is the largest improvement. It can prompt the user to create web extraction projects step by step. It contains a number of commonly used web scraping templates for building various projects to scrap the page content from websites of different structures.