What's new in Benchmark Factory (formerly Benchmark Factory for Databases) 7.0 Build 221
Jul 25, 2014- New Job Wizard:
- This release includes the New Job Wizard, a completely re-designed wizard that easily guides you through the process of creating a new job.
- Select the type of test to run from the drop-down list (after selecting Workload in the left pane).
- This user-friendly wizard simplifies the process of creating a new workload to test. Click on a test/step in the left pane, then specify the options in the right pane.
- To open the New Job Wizard, click New in the main toolbar.
- You can modify existing jobs by opening the wizard in Edit mode. Select a job in the Jobs View pane and click Edit Job in the main toolbar.
- After specifying the workload for a job, specify an agent or agents by selecting Agent in the left pane.
- The first step when creating a new job is usually to select a connection. Select Database Under Test in the left pane to specify your connection.
- The Database Under Test page of the wizard displays database and host information. You can edit the information and add user-defined property/value combinations.
- Agents:
- To setup agents, you can use the Agent page of the Settings dialog (Edit | Settings | Agent).
- You can now deploy remote agents from the Benchmark Factory console. Enter the IP address or name of the remote machine. Then Benchmark Factory allows you to launch the installer, install to the remote machine, and configure agents.
- Note: You must be able to connect to and have the necessary privileges/account to install on the remote machine.
- When you run a job, agents now start and stop automatically.
- Jobs View:
- Run Multiple Jobs: The Benchmark Factory Console has been enhanced to allow you to execute multiple tests against multiple targets at the same time.
- Test Results:
- The Results page is now the default page for a job selected in the Jobs View pane. In addition, it has been redesigned to allow you to more easily find the results of a test execution.
- To view test results, select a job in the Jobs View pane. The Results page displays in the right pane.
- You can export one or more test results to Word or Excel, or print the results. Select a run and click the Export Report Summary to Word, Export Report to Excel, or Print Report Summary button.
- You can export test results as a zip file containing an XML and other files for reproducing test results. Select a run and click the Export Test Result button.
- You can import test results from a zip file exported from Benchmark Factory. Click the Import Test Result button.
- To open the test results in Run Reports, select a test run and click Show Detail Results (or Compare Test Results if multiple test runs are selected). Use Run Reports to view detailed test results.
- You can select a test to set as the baseline to easily compare other run results against this baseline. Select a test run and click Set as Baseline. The baseline icon appears in the Run column of the baseline run.
- A new Compare Results page has been added to allow you to quickly compare test results for different runs. Select two or more runs to display test results side-by-side for easy comparison.
- Select Automatically compare to baseline to compare the selected run to the baseline.
- Industry Standard Benchmark Testing:
- Checking Scale Factor: Benchmark Factory is able to check the Benchmark Scale factor of existing benchmark tables against the new Benchmark Scale requirement before executing a test. In this release, you can easily turn the checking process on or off in Edit | Settings | Benchmark.
- Refresh data using backup sets: You now have the option to create and use backup tables to quickly recreate data for subsequent test runs. In the New/Edit Job Wizard, in the Options tab (under Test Options tab), select Refresh data using backup sets. You can also specify this as the default setting for new jobs in Edit | Settings | Benchmarks.
- Connections:
- You can create, edit, and manage connections using the New Connection and Edit Connection buttons on the main toolbar.
- Or use the new My Connections tab which displays in the lower pane of the Benchmark Factory console and lists all your currently-defined connections. You can edit, manage, and create connections from this tab as well.
- Benchmark Factory now collects and displays database and host server information for each connection. The information is displayed in the Environment tab of the Connection dialog.
- You can edit the information or add user-defined property/value combinations.
- You can also access and edit connection information through the Edit Job Wizard (or the New Job Wizard). Select a job in the Jobs View pane and click Edit Job. In the Edit Job Wizard, select Database Under Test.
- Benchmark Factory Settings:
- In this release, the Settings dialog has been redesigned and includes an easy-to-use tree-list navigation.
- In addition to the many new Settings options already mentioned, the following new options are also available in the Settings dialog:◦
- In Edit | Settings | Statistics, you can specify the following statistics options. (In addition, you can also specify these options for a test in the Test Options | Advanced tab of the New/Edit Job Wizard.)
- In Edit | Settings | Oracle, you can specify default settings for Statspack and AWR/ADDM statistics collection and reporting. (In addition, you can also specify these options for an Oracle connection.)
- General:
- Benchmark Factory now collects and reports IOPS for supported DBMS.
- ADDM report is now generated.
- You can now specify a location to save Captures and Result Logs. Select Edit | Settings | General to specify a new location.
- You can now instruct Benchmark Factory to terminate the job, not just the test, after the first test error. In the New/Edit Job Wizard, select the Job Setup tab, and then select Stop job after first error in the General tab. You can also specify this as the default setting in Edit | Settings | General.
- Resolved Issues and Enhancements:
- Added an option to the Create Objects test/step to Perform data refresh if tables exist.
- When defining an Oracle workload capture, the default setting for Capture Method is now Oracle Fine-Grained Access Control instead of Oracle Trace Files.
- The workload capture option Oracle Fine Grained Auditing (FGA) has been renamed to Fine Grained Access Control (FGAC).
- When defining an Oracle workload capture, in the Apply Privileges page, the User field is now blank to indicate that you can choose any user account that has the necessary privileges.
- Oracle: An issue that prevented the Statspack report from being generated during an Oracle Capture and Replay on a 12c non-container database has been resolved.
- SQL Server: An issue that unexpectedly removed the transactions executed by sp_executesql from the imported SQL Server trace when attempting to replay a SQL Server workload has been resolved.
- SQL Server: When importing a trace file from a SQL Server capture, simple TSQL call is also imported.
- If Benchmark Factory unexpectedly closes, any jobs that were added during the previous session are now preserved.
- The Benchmark Factory Console has been enhanced to allow you to execute multiple tests against multiple targets at the same time.
- When a license key has a seat count of "0," the Licensing dialog now correctly displays the quantity as "unlimited."
- If the user runs a SQL Server replay containing more user scenarios than the license allows, and the Agent is already connected to the console, an alert message displays.
- An issue that caused an "Unhandled exception" error (followed by the application unexpectedly closing) when attempting to add a new performance counter from "local computer" for "all instances" is resolved.
- Support testing DB2 databases against the TPC-E industry standard benchmark.
- DB2: TPC-E Benchmark test running on a DB2 database no longer errors out with overflow error with scale factor 15 or higher.
- An issue with generating data on the E_TRADE_TYPE table with scale factor greater than one is resolved.