MySQL Utilities Changelog

What's new in MySQL Utilities 1.6.5

Apr 19, 2018

Bugs Fixed:
Corrected a problem when using a --exec-before script that failed causing the mysqlfailover utility to fail silently. (Bug #25030151)
Added corrections to the message text of some utilities to eliminate typographical errors. (Bug #24742356, Bug #83173)
The mysqldbcopy utility failed to copy views that refer other views in the list of databases to be copied. (Bug #24657793)
Corrected an issue when using anonymous hosts. (Bug #24624716, Bug #82636)
The mysqlfrm utility failed to parse some .frm files properly. This fix ensures that the utility can parse tables with many columns in the field list. (Bug #24613794, Bug #82907)
The mysqldbcopy utility failed to copy tables with a primary key set to zero and with AUTO_INCREMENT applied. This fix enables the copy and export of auto-increment columns with an initial value of zero. For export, the user is warned to turn on the NO_AUTO_VALUE_ON_ZERO mode for the import and suggested SQL statements to enable the mode are provided. For copy, the mode is turned on automatically and turned off after the copy. (Bug #23598948, Bug #81866)
Permits IPv4 address parsing to include values of 0-255 rather than 0-254. Thanks to Erol Guven for the patch. (Bug #23578714, Bug #81834)
Tables with BLOB data in fields marked NOT NULL restricted the operation of the mysqldbcopy utility. This fix enables the utility to copy these tables by temporarily altering the columns on the destination to remove the NOT NULL option and then to restore it after the copy. (Bug #23335753, Bug #81553)
Corrected an issue with reading strings from configuration files when run with Python 2.6. (Bug #22738248)
Corrected the error message stating the --discover-slaves-login option is not used with the failover command for the mysqlrpladmin utility. (Bug #22616847, Bug #80082)
Corrected the problem when socket files are used in replication utilities. Previously, the port defaulted to 3306 instead of the actual port for the servers. The code now retrieves the port from the connected servers so that connections of slaves are directed to the correct port. (Bug #22543116)
Added context to errors resulting from server startup for the mysqlserverclone utility. Messages recommend using the -vvv option to find the actual error from the server startup sequence. (Bug #22457829)
The mysqlfrm utility failed when the --verbosity option was used. This fix replaces the --verbosity option with --verbose, the correct option name, and uses it consistently in error messages and code. (Bug #22254445, Bug #79382)
The mysqlserverinfo utility required connection parameters to show all MySQL servers running on the local host. This fix removes that restriction when the --show-servers option is specified. (Bug #22215626, Bug #79135)
Corrected a problem in the mysqldbimport utility where specifying the --skip option with a valid type failed to skip the intended object or objects during the operation. (Bug #22155217, Bug #79128)
The mysqldbcopy utility failed to copy grants for anonymous users. (Bug #22093518, Bug #78940)
Improved the --quiet option to silence all informational messages for the mysqldbcompare utility. (Bug #22093453, Bug #78938)
Corrected a problem using a single database name in the mysqldiff utility when comparing the same database on two servers. (Bug #21764003)
The DEB packages did not create the required /usr/share/pyshared/mysql directory. (Bug #21626041, Bug #78054)
Added error handling to ensure only one slave is specified for the --slave option for the mysqlreplicate, mysqlrplcheck, and mysqlrplms utilities. (Bug #21497923)
Enabled the use of the --exclude option for the mysqlrplsync utility. (Bug #21270853, Bug #77359)
Corrected a logic error when detecting and setting READ_ONLY mode on the candidate server during promotion for failover. (Bug #21201454, Bug #76522)
When the mysqldiff utility compared columns for indexes, it failed to take into account cardinality changes. That is, if the cardinality of one column differed from the corresponding column in the compared table, the index was considered different. This fix eliminates unnecessarily changes to indexes by the utility when cardinality differences are detected. (Bug #20857151, Bug #76619)
Removed password() function usage in the MySQL Utilities library for MySQL 5.7.6 or later. (Bug #20807296)
The mysqlfailover utility did not remove the failover_console table correctly on exit when run with the --daemon=start option. (Bug #20524886)
Improved the --force option for the mysqldiff utility to allow checks to continue when missing objects are detected. (Bug #20511130, Bug #75822)
Removed an extraneous difference in GRID format output of the mysqldbcompare utility when column sizes differ. (Bug #18763591)
Corrected PEP8 warnings and errors on the utilities code base. (Bug #18755438)
Corrected an issue in which the mysqldbexport utility produced output with inconsistent table and database names. (Bug #18733683)

New in MySQL Utilities 1.6.3 Beta (Jul 27, 2016)

Bugs Fixed:
The - exclude option for the mysqldbexport command did not function.
The mysqlfailover and mysqlrpladmin utilities did not function with MySQL 5.7 Servers. During the failover process, they failed with the following error: "Errors: Missing gtid_executed system variable."
The mysqlrplsync and mysqldbcopy utilities did not function with MySQL 5.7 when used in a topology with a master. The problem was due to table locking, and began occurring when session_variables was moved to PERFORMANCE_SCHEMA in MySQL 5.7.6. The master UUID is now retrieved before locking the tables.
Several man pages were missing from the RPM package.
The mysqldbcompare utility now checks if MySQL servers are aliases.
Using the mysqlauditgrep utility with the --format option would throw an exception when the audit log file contained Unicode characters, and when the format was changed to something other than the default RAW. The error looked similar to "UnicodeEncodeError: 'ascii' codec can't encode characters in position".
With the mysqldbcompare utility, using the same connection information for both --server1 and --server2 with the --all option would raise an AttributeError error.
The mysqldbexport utility's --exclude option was ignored for data when passing in either the data or both value.
Several utilities (the daemon utilities, mysqlfailover, and mysqlrplms) created pid files with the default permissions on *nix systems (0777). The permissions are now more restrictive.
MySQL Utilities failed to execute when the sql_mode option included "ANSI_QUOTES".
Executing the mysqldbcompare utility to find differences between two databases on different hosts could fail with an error similar to "Unread result found".
The MySQL utilities did not function with .mylogin.cnf as of MySQL Server 5.6.25 and 5.7.8 due to my_print_defaults masking passwords by default.
The Windows MSI 64-bit package, by default, installed in "Program Files (x86)" on a 64-bit system, instead of "Program Files".
The mysqldiskusage utility incorrectly calculated disk usage, leading to an inaccurate result.

New in MySQL Utilities 1.6.2 (Jul 27, 2016)

Functionality Added or Changed:
A new optional --inherit-level parameter was added to the mysqlgrants utility. It specifies the inheritance level of the grants shown, and is set to either global (default), database, or object.
Starting with MySQL Utilities 1.6.2, MySQL Fabric is no longer included as part of the MySQL Utilities release. They are now separate MySQL products with separate release cycles.
Bugs Fixed:
The mysqldbcompare utility's --all option now ignores the internal sys schema.
The mysqlimport utility would fail with an "Invalid Statement" error (errno 1694), when executed against MySQL server 5.7.7.
The mysqluserclone utility did not properly copy global privileges with --include-global-privileges enabled, as it attempted to grant privileges to the source user instead of the destination user.
The mysqlrplsync and mysqldbcompare utilities now function correctly when the sys schema is present; a schema that is added by default as of MySQL server 5.7.7. In other words, the sys schema is no longer compared/synced as it is unique to each host.
The mysqldbcopy utility failed to copy a database that contained permissions to a user not present on the destination server when NO_AUTO_CREATE_USER was enabled. This is because mysqldbcopy attempted to create the user implicitly using the GRANT statement, an operation that does not work with NO_AUTO_CREATE_USER. Users are now explicitly created before the GRANT statement.
Related, the NO_AUTO_CREATE_USER sql_mode is enabled by default as of MySQL server 5.7.7, and will become default behavior in a future release.
The mysqlindexcheck utility would accept options and execute when an invalid --server option was passed. A valid server connection is now required. Also, the utility would report "Using a password on the command line interface can be insecure." even when a password was not used.
The mysqldbexport utility required SELECT privileges to the mysql database when such permissions were not needed, even when passing in --skip=procedures,functions.
When using the mysqluc console, pressing Home would print the 'H' character instead of moving the cursor to the beginning of the line, and pressing End would print the 'F' character instead of moving to the end of the line.
Pylint tests were failing with the latest versions of pylint and pep8.
Several MUT tests were failing with MySQL server 5.7.6.
MySQL Utilities would hang when a long query using Connector/Python 2.1.2 was killed before completion. The underlying bug was fixed in Connector/Python 2.1.3.
The MySQL Utilities test suite now tests the strict SQL mode.
The maximum length of mysql.failover_console.host was set to 30 characters. The mysqlfailover utility failed to start when host names were longer than this. The new maximum length is 255.
Some replication utilities, such as mysqlfailover and mysqlrpladmin, would crash with MySQL server 5.7.6 (and above) because a new column (Channel_name) was added to the mysql.slave_master_info table.
The utilities were updated to not fail when additional columns are added in the future.
The mysqldbexport and mysqldbimport utilities failed to correctly export/import databases using non-SQL formats like CSV, if the database contained a table with compound keys. This issue did not happen for the SQL format (default), it only occurred for the other available formats, such as CSV and TAB.
A rpl_admin_gtid test set the gtid_purged variable on the master server to create a controllable scenario where a slave server was missing GTIDs from the server. However, this was a faulty behavior from the server that was fixed in 5.6.22 and 5.7.6 via Bug #19012085.
The mysqldbimport utility was unable to import multiple databases from a single file if the file format was not SQL, such as CSV.
Under some conditions, some commands that manipulates privileges can now be executed/invoked remotely.
The mysqldiff failed to generate correct ALTER statements that differentiated between DEFAULT NULL and DEFAULT ''.

New in MySQL Utilities 1.6.1 Alpha (Jul 27, 2016)

Functionality Added or Changed:
The following utilities were added: mysqlbinlogpurge, mysqlbinlogrotate, and mysqlslavetrx.
Bugs Fixed:
Generally, the code was improved to be more MySQL 5.7 compliant.
All CREATE or ALTER TABLE commands in the Utilities, and Utilities that generate them, were updated to use fully qualified table names with foreign keys.
As of MySQL 5.5.41 / 5.6.22, there is a new behavior with foreign keys. You must now explicitly state the schema of the table that the foreign key points to. Previously, if missing, it was assumed that the schema of the table pointed to by the foreign key was the schema of the table being defined, but this is no longer true as of MySQL 5.5.41 / 5.6.22.
The _parse_grant_statement method from the common/user.py module incorrectly parsed MySQL 5.7 GRANT statements if they were after 'IDENTIFIED BY PASSWORD' and followed by a hash of the password. The hash is now seen as optional.
The MySQL Utilities did not correctly handle spaces in the PATH to the MySQL server, which is common on Windows. These paths are now surrounded by quotes.
The mysqlserverclone utility did not function properly with MySQL 5.7. It would bootstrap the server, load the SQL files, and relaunch the server, but it would not connect afterwards.
The MUT test named binlogmove_privileges failed on some servers due to the unexpected rotation of relay log files on the slave. All slaves are now stopped as soon as the test relay log files are created, as to prevent log rotation.
The argument parser validation and error handling was improved for the mysqluc utility.
The results regarding index redundancy shown by the mysqlindexcheck utility were not correct for FULLTEXT nor HASH type indexes.
For FULLTEXT indexes, the order is not relevant, nor is the number of columns. A FULLTEXT index i(a,b) is redundant of a FULLTEXT index i2(b,a), and is also redundant of a FULLTEXT index i3(a) or FULLTEXT index i4(b). In other words, a FULLTEXT index is redundant if its columns are a subset of another FULLTEXT index.
As for HASH type indexes, one index in considered redundant of another index if it contains the exact same columns with the exact same order. In other words, two HASH type indexes are redundant only if they are duplicates.
Generated logs now contain the version of the MySQL Utilities and connected MySQL server.
On Windows, the test suite (MUT) failed to kill the spawned servers after running the tests, if the path for the base MySQL server included spaces.
When a table contained a composite Primary Key, the mysqldbcompare utility generated INSERT/UPDATE/DELETE statements that omitted the "AND" clause between each field.
When comparing databases with the mysqldbcompare utility, quote characters within resulting SQL transformation statements were not escaped. As a result, the statements were not valid and could not be executed directly in a client tool, as intended.
MySQL Utilities now reflects the SSL behavior introduced for the MySQL client in MySQL 5.7.3. Setting --ssl=1 now causes the MySQL connection to fail if an encrypted connection can not be established.
The mysqldbcopy utility was not able to copy table rows that contained single quotes in text columns.
The conversion to CSV performed with the --format=CSV option for several utilities was applying an incorrect line terminator on POSIX (non-Windows) systems. In particular, "rn" was used instead of "n".
The mysqldbcompare utility failed to execute when a large invalid value was passed into the --span-key-size option.
In MySQL, a BTREE index A is considered redundant of a BTREE index B if and only if index A is a leftmost prefix of index B. Having a common partial prefix is not enough. The mysqlindexcheck utility had an issue and was considering two BTREE indexes as redundant if they had a common partial prefix. mysqlindexcheck now correctly checks if a BTREE index is a leftmost prefix of the other index.
The mysqldbcompare utility executed the full algorithm twice to search for data differences when the --show-reverse option was used, instead of only repeating the part that generated the differences in the reverse direction. This had a notable performance impact for larger databases.
The mysqldbimport utility did not issue an error when a non-existing file was specified while multiprocessing support was enabled. Also, without concurrency (the multiprocess option enabled), a verbose Traceback was printed instead of a more readable error message. Now, a friendly error message is generated for both cases.

New in MySQL Utilities 1.6.0 Alpha (Jul 27, 2016)

New in MySQL Utilities 1.5.6 (Jul 27, 2016)

New in MySQL Utilities 1.5.5 (Jul 27, 2016)

Functionality Added or Changed:
In the Fabric configuration file, the backup_user and restore_user MySQL user accounts were added to the [servers] section, and the [client] section (along with its user option) was removed altogether. The Fabric configuration file is no longer used by MySQL client programs. (Formerly the file was used as the defaults file for the MySQL clients via the --defaults-extra-file option.)
For additional information about setting up these MySQL accounts for Fabric, see Create the Associated MySQL Users.
For Fabric, an example script was added to the source packages to help provide an HA solution using a single Fabric node. For related documentation, see http://dev.mysql.com/doc/mysql-utilities/1.5/en/fabric-pacemaker.html.
Bugs Fixed:
The mysqldbcompare utility's --all option now ignores the internal sys schema.
The mysqlimport utility would fail with an "Invalid Statement" error (errno 1694), when executed against MySQL server 5.7.7.
For Fabric, executing "server clone" would throw the following error; "'NoneType' object has no attribute 'user'".
For Fabric, executing >build_docs without sphinx installed would emit an unclear error message.
For Fabric, errors were emitted for commands executed locally using the dispatch method that also contained compulsory arguments.
With Fabric, MySQL server 5.7.8 and newer would not function as a state store, and would instead emit an error similar to "Unknown column 'group_id' in 'where clause'.
For Fabric, the default MySQL port is now passed in with the server_address when the port number is not explicitly defined.
In the MySQL RPC protocol, the MySQL Fabric process would spin in the "recv" after a connector closed the connection, which caused excessive (100%) CPU use. The code now detects when an empty chunk is returned from "recv", meaning that nothing else will be transferred from the client and that the socket is closed.
In addition, possible header fragmentation is now avoided when the process would hang when attempting to read 4 bytes.
The mysqluserclone utility did not properly copy global privileges with --include-global-privileges enabled, as it attempted to grant privileges to the source user instead of the destination user.
The mysqlrplsync and mysqldbcompare utilities now function correctly when the sys schema is present; a schema that is added by default as of MySQL server 5.7.7. In other words, the sys schema is no longer compared/synced as it is unique to each host.
The mysqldbcopy utility failed to copy a database that contained permissions to a user not present on the destination server when NO_AUTO_CREATE_USER was enabled. This is because mysqldbcopy attempted to create the user implicitly using the GRANT statement, an operation that does not work with NO_AUTO_CREATE_USER. Users are now explicitly created before the GRANT statement.
Related, the NO_AUTO_CREATE_USER sql_mode is enabled by default as of MySQL server 5.7.7, and will become default behavior in a future release.
The minimum Connector/Python version increased to 2.0.4 or 2.1.2.
The mysqlfabric client would exit with a "Connection reset by peer" error if a thread was not available when using the XML-RPC protocol, although subsequent commands would typically succeed.
MySQL Utilities would hang when a long query using Connector/Python 2.1.2 was killed before completion. The underlying bug was fixed in Connector/Python 2.1.3.
With Fabric, programming errors could leave some results unread and, by consequence, the associated connection cannot be used to process other requests until the result was read. This issue could break threads that had an associated connection as they could not process further requests after the first "unread results found" error. (Bug #20415833, Bug #75589)
Some replication utilities, such as mysqlfailover and mysqlrpladmin, would crash with MySQL server 5.7.6 (and above) because a new column (Channel_name) was added to the mysql.slave_master_info table.
The utilities were updated to not fail when additional columns are added in the future. (Bug #20344064)
A SIGUSR1 handler was added to MySQL Fabric. If a SIGUSR1 signal is sent to the MySQL Fabric process, the stack trace of all threads are now sent to the current logger, if there is one, otherwise to the stderr stream.
An empty handler for the SIGINT signal was added just in case the MySQL Fabric process is killed by mistake after pressing Control + C. To stop the MySQL Fabric process, you can either execute "mysqlfabric manage stop", or send it the SIGKILL signal.
Also, stderr was not redirected when the MySQL Fabric process was daemonized, so the process would die if someone tried to dump a stack trace. Now, 'stderr' is redirected to 'os.devnull' (i.e. '/dev/null' for POSIX).
With Fabric, an interrupted test suite caused future executions of the test suite to fail due to test database remnants. The test suite now drops existing test databases and test users before creating new ones.
With Fabric, the test suite now checks if the required environment variables, such as MYSQLDUMP and MYSQLCLIENT, are set before proceeding.
The [client] section of the Fabric configuration file was removed, to avoid confusion with the related [servers] section.

New in MySQL Utilities 1.5.4 (Jul 27, 2016)

New in MySQL Utilities 1.5.3 (Jul 27, 2016)

New in MySQL Utilities 1.5.2 (Jul 27, 2016)

Bugs Fixed:
Added the missing README and LICENSE text for the commercial packages.
For Windows, the Utilities MSI distribution package was setting the wrong permissions for the generated installation directory, which included all permissions granted for the 'Users' group. Now, the only permission set here for the 'Users' group is read-only.
The failure detector was specifying a set of groups as lockable objects, and a failover operation could potentially run while a group was being updated by another operation which could lead to unpredictable results.
A debugging message was being sent through the Fabric RPC protocol.
Due to licensing conflicts between OpenSSL and GPL, SSL support is not included in the community edition for Windows.
Fabric behaved inconsistently with respect to sharding and non-existent tables.
For Fabric, Connectors inserting large amount of data before the TTL expires, after a resharding metadata update, could cause inconsistencies between the sharding definition and the data on the shard.
Separate CHANGES and README files were added to the commercial distribution.
Fabric could return an incorrect "lower_bound" length, depending on the character set.
The --disable-binary-logging flag of the mysqldbcompare utility did not function properly. It always required SUPER privileges in cases where the server had binary logging enabled, it was enabled internally even when not in use, and it threw an error when attempting to generate SQL statements for the differences between tables if one of the rows had columns with NULL values.
Also, the ability to call the to_sql method for each column was added, which is enabled by default for the CSV output type.
The mysqlserverclone utility did not check for a low amount of free space before it created a new instance of the server. It now errors out if 120+ MB of free space is unavailable, but this check can be ignored by also passing in the --force option.
Additionally, the error handling for failed new data directory creation was improved to show additional details about the failure.
When copying views that depended on sub-views that were not yet copied, some utilities such as mysqldbcopy failed. The dependency related code was fixed to solve this problem.
The policy command for the mysqlauditadmin utility failed when applied to recent MySQL server versions (such as MySQL 5.6.20 and 5.7.5). This is because audit_log_policy is now a READONLY variable.
The utilities did not parse all types of GRANT statements, such as those with PROXY privileges and stored routines. The utilities could also fail when objects contained backticks.
The mysqldbcompare utility failed when comparing two databases while using the SQL difftype.
Commands accept either a UUID or an address (i.e. host:port) to specify a server. The code was ignoring that a wrong address could be provided, and in consequence the execution could hang. The address is now validated before it is used.
If the server belongs to a group, the address is checked against the information stored in the state store.
If the server is not part of a group, Fabric will try to access the server until a timeout is reached. The default value for this timeout is defined through the option "unreachable_timeout" in the "servers" section in the configuration file, and it affects the following commands:
server lookup_uuid address [--timeout=NONE] ...
server clone group_id destn_address [--timeout=NONE] ...
group add group_id address [--timeout=NONE] ...
(Bug #19172889, Bug #73233)
Some of try/exception blocks were catching the DatabaseError exception and raising a new exception without propagating information from the original problem.
The error messages were improved by adding information about the server where the error originated, and the caught DatabaseError information is merged into the appropriate objects.
The --force option was removed from the mysqlfabric utility. It was removed to help alleviate confusion about its purpose. Its functionality was replaced by clearer status messages that are always printed out.
The --skip-innodb option is used by the utilities, but this option is ignored in MySQL 5.7+. A warning is now issued for this scenario.
The mysqldiff utility's --skip-table-options option would fail when comparing two tables with different column ordering, or when also setting the --difftype option to SQL.
The RPM installation failed to check for the required Connector/Python version, and having the incorrect version installed caused "mysqlfabric manage setup" to fail.

New in MySQL Utilities 1.5.1 RC (Jul 27, 2016)

Functionality Added or Changed:
Provisioning support was added to Fabric, by supporting the Nova Interface, which is responsible for managing compute instances to easily expand and shrink the number of servers in a group when MySQL Fabric is deployed within an OpenStack Cloud Provider. This introduces commands for the provider, machine, and snapshot groups.
Fabric now supports DATETIME and UTF-8 strings as sharding keys in range-based sharding.
The internal interface between the protocol classes and the Fabric core was improved to be more consistent.
Fabric now supports the MySQL Client-Server Protocol. Only connectors and clients supporting PROTOCOL41 and MULTI_RESULTSET are allowed. This preferred interface is an alternative to the current XML-RPC interface.
Bugs Fixed:
The specification and control files were updated to reflect the Connector/Python 2.0.0+ dependency.
Added support for Connector/Python 2.0.0 and higher.
The mysqlreplicate and mysqlrplms utilities would generate an exception when the optional --rpl-user was not passed in.
The check for an errant transaction was not performed for mysqlrpladmin's switchover command, it only performed for the failover command. Both commands now check for errant transactions.
The errant transaction check requires all servers in the topology to have GTID enabled, otherwise a warning is issued indicating that the check cannot be performed for the switchover command.
When a login-path (or config-path) was used for server connection values, and the .mylogin.cnf file was not found, an AttributeError was issued instead of a missing file error.
On OS X Mavericks, the testing suite (MUT) failed to kill mysqld processes as MUT could not identify the MySQL server ports.
On Windows, the mysqlfabric utility searched for the fabric configuration file outside of the installation folder.
The default configuration file location is C:Program Files (x86)MySQLMySQL Utilities 1.4.4etcmysqlfabric.cfg.
Additionally, the MySQL Utilities section in the Windows Start menu now contains a link to the fabric configuration file.
With fabric, the --daemonize parameter was not recognized when passed in with manage start.
In the mysqluc environment, pressing tab after typing a "q" would incorrectly suggest the term "qxit" instead of "quit". This now returns "quit", which is an alias for "exit".
On Linux, the backspace key was not functioning in the mysqluc environment, therefore the entered text could not be changed.
The Debian packages checked for a Connector/Python 1.0.9+ prerequisite instead of requiring Connector/Python 1.2.1+.
Failed MySQL server connection error messages now return the actual error returned from the failed connection, instead of a custom (and more generic) error. This new behavior is more standard with how other MySQL client applications behave.
With the --discover-slaves-login option, if there were connection or configuration problems with some of the slaves, then those slaves were skipped with only a generic error message being issued. This behavior did not change when also passing in --verbose.
A simple warning about the connection/configuration problems is now generated during the discovery process, and it references including --verbose to see additional information. The --verbose option generates specific information for each slave, and includes their respective errors.
When running the mysqlserverinfo utility as root against an offline MySQL server, it failed to start a read-only instance of the server and emitted the fatal error "Please read "Security" section of the manual to find out how to run mysqld as root!". This occurred because the mysqld option "--user=root" was not passed in, which is mandatory when starting mysqld as root. The user_name that executes the utility is now detected and added as --user=user_name when starting the read-only MySQL server.
Additionally, error checks were added that validate the --basedir and --datadir options to check for their existence and accessibility.
Descriptive text for the --no-headers option was improved to clearly indicate which formats it supports. For example, it does not support the "vertical" format.
The minimum required Connector/Python version in the source code was defined as 1.0.9 instead of 1.2.1.
When cloning a user, the mysqluserclone utility did not check if the user of the destination server had the required privileges for a successful operation
The mysqluserclone utility was throwing an unhandled exception when the --list option was used with incorrect login credentials. The error message is now emitted.
The mysqluc utility would suddenly exit when a command was wrongly formatted, like when a closing quote was missing. This action now generates a warning instead of exiting from the console, which allows the command to be edited.
When passing in the --dump option to the mysqluserclone utility on a user with the hostname '%', the output (each grant) would be printed twice.
Error messages for the mysqluserclone utility were improved by making them more explicit in terms of what might be missing.
With fabric, sharding prune_shard operations would hang when performed on a very large number of records.
The default value was removed from the --rpl-user option for the mysqlreplicate and mysqlrplms utilities. Before, the default user/pass value was "rpl:rpl".
A warning is now emitted whenever a password is entered into the command line. When testing, this warning can be disabled by setting self.mask_global to false.
The mysqlfailover utility caused excessive CPU load when executed in the console without passing in --daemon. This high CPU usage was due to the process waiting for user input.

New in MySQL Utilities 1.5.0 Alpha (Jun 30, 2014)

New in MySQL Utilities 1.4.3 (May 27, 2014)

Bugs Fixed:
Due to the several changes in the high-availability and sharding sub-systems, the automatic routine to automatically find the lockable objects for a given procedure was not always valid. The consequence was that some procedures may have tried to update the same group concurrently when they should have been serialized. (Bug #18712020, Bug #72553)
When executing "mysqlfabric manage setup", some of the generated "CREATE TABLE" statements were explicitly requesting the InnoDB engine and ignoring the default-storage-engine option, thus preventing use of NDB as a state store. (Bug #18648779)
Any empty password definition ("password = ") in the configuration file would emit errors. (Bug #18524482, Bug #72149)
The FailoverDaemon class from the failover_daemon module was refactored to extend the Daemon class, so that the duplicated code was removed. (Bug #18546744)
Remote commands were authenticating the request twice, both before and after the command was sent to Fabric. This meant unnecessary requests were made to the state store. (Bug #18477189)
Executing "mysqlfabric manage setup" would sometimes halt with a connection error due to a short timeout. (Bug #18458461)
Commands failed if the "protocol.xmlrpc.user" option was not provided in the configuration file or from the command-line. Now, the user "admin" is the default. (Bug #18459012)
A "group add" with a server as "localhost:32274" caused Fabric to hang, as 32274 is the port used by Fabric. (Bug #18454737, Bug #72119)
A "PROMOTE" after "RESET MASTER" would cause Fabric to hang. Now, "SHOW SLAVE STATUS" is used to check whether the slave has processed all of the entries in the relay log. (Bug #18454679, Bug #72118)
The mysqldbcompare utility was not correctly checking differences for different rows internally associated to the same span key. Only the first difference was displayed. (Bug #18477410)
Optimizations to the mysqldbcompare utility include:
A new step was added to the data consistency check, now executing a CHECKSUM TABLE first (which is faster when no differences are expected). The current comparison algorithm is only execute if this preliminary full table checksum fails. A new --skip-checksum-table option was added to skip this new step that otherwise is executed by default.
The BINARY type is used instead of CHAR to store hash values in a more optimized way.
Unnecessary keys were removed from the temporary table (including the primary key).
ENGINE=MyISAM is now used for the temporary compare table.
(Bug #18483844)
The temporary table created by the mysqldbcompare utility was missing an index for the 'span' column, meaning that the "_COMPARE_SUM" and "_COMPARE_DIFF" queries were doing full table scans. (Bug #18483844)
Some Fabric commands, such as "server.set_status" and "group.promote", accepted a reference to a MySQL server's uuid but not a reference to an address and port. (Bug #18440617, Bug #72094)
The mysqlrplms utility did check the MySQL server version when executed on MySQL server 5.7. (Bug #18434209)
With fabric, the pylint_tests script mixed the output from both pep8 and pylint results. A new --tester option was added that allows the tester to be chosen, which can be either "pylint", "pep8", or "all" (the default). (Bug #18383839)
Invoking the mysqlfabric utility with the wrong number of parameters would display general usage help for the utility, but it now shows help for the specified command. (Bug #18370958)
The mysqlrplsync utility was allowing the --exclude option to take an empty value, and the user was not notified. This now generates an error. (Bug #18381748)
The mysqldbcopy utility threw an internal Python exception when --rpl=slave was passed in without specifying --rpl-user. (Bug #18338321)
The "__name__ == '__main__'" attribute check was added to all scripts that can be executed as a standalone program. (Bug #18376793)
The mysqldbcopy, mysqldbimport and mysqldbexport utilities were not fully functional for all possible data types. Prefixes such as TINY, SMALL, MEDIUM, and BIG were not being taken into account when verifying the column data type.
This issue was only present when the source server did not equal the destination server. In other words, when copied and not cloned. (Bug #18373190, Bug #71890)
The mysqldbexport utility would export BLOB rows using two statements; an INSERT followed by an UPDATE. This did not function if the exported table did not have at least one "UNIQUE NOT NULL INDEX", since UPDATE statements cannot uniquely identify each row. Now, they are exported using a single INSERT statement. (Bug #18373190, Bug #71890)
dump.shard_* did not take into account disabled shards, and therefore reported disabled shards through the dump interface. (Bug #18259479, Bug #71701)
The mysqlserverinfo utility threw an internal Python exception when passing a non-existent server to --server. (Bug #18262507)
The mysqslindexcheck utility was considering a PRIMARY key as redundant when this key was compared to a unique index with more columns than the PRIMARY key, which caused the DROP statement to be displayed as "None;" with the --show-drops option. In this scenario, the index key is now marked as redundant instead of the PRIMARY key. (Bug #18281513, Bug #71745)
The mysqlauditgrep utility did not support changes made to the audit log format in the "NEW" format. Both "NEW" and "OLD" formats are now supported.
The audit_log_format option defaults to "OLD" in MySQL server 5.6, and "NEW" in MySQL server 5.7. (Bug #18314951, Bug #71838)
An indexcheck option was added to the mysqlindexcheck utility. It reports on tables without a PRIMARY or UNIQUE key. Previously, this information was only available with a high verbosity level (-vv). (Bug #18241174, Bug #71690)
mysqldbcompare failed to properly compare the same views on different schemas. This caused the objects to be seen differently. (Bug #17864081, Bug #71022)
Because each xmlrpc session requires a database connection, it is possible to exceed the maximum number of connections in the backing store. The failure detector did not function properly when the number of connections was exceeded, and as such it could not promote a new server to the master. This scenario now generates an exception when fabric starts with the built-in failure detector is enabled. (Bug #17747197)
The mysql/utilities/common/parser.py script would stop processing when a blank line was encountered. However, the General Query Log may include multi-line statements that include blank lines. (Bug #71851)

New in MySQL Utilities 1.4.2 RC (Apr 1, 2014)

Functionality Added or Changed:
The mysqlrplsync utility was added, which checks data consistency between servers in a replicated setup.
The mysqlrplms utility was added, which provides round-robin multi-source replication (a slave server continually cycles through multiple masters in order to store a consolidated data set).
Bugs Fixed:
With fabric, the scheduler did not check and determine which procedures needed to be notified after releasing the procedure that was blocking them. (Bug #18454582, Bug #72117)
With fabric, failover was not handled properly when both the global and shard groups had a failed master. (Bug #18403885, Bug #72016)
The mysqlrplsync utility would crash when the --exclude option was used, and at least one database was available to be checked. This issue was due to an incomplete check when determining which tables to exclude from the slaves. (Bug #18388846)
The MySQL Utilities testing suite (mut) now tests if ports are available before assigning them to a test. (Bug #18339351)
MySQL Server UUIDs are now handled in a case-insensitive way, more precisely when checking if GTIDs belong to the master. This helps accommodate MySQL Server 5.6.9, which could change the case for the server_uuid variable. (Bug #18275566)
With Fabric, attempting to destroy a group when the group was used in the shard sub-system would output an unfriendly and unclear error. Fabric now checks if the group is used in the shard sub-system before trying to destroy (remove) it, and a clearer error is emitted if it is in use. (Bug #18280004, Bug #71766)
The mut rpl_admin_gtid test result varied from host to host, due to using the RESET MASTER statement without first stopping all the slaves. This resulted in undefined behavior, which led to inconsistent rest results. (Bug #18203946)
A typo was fixed in the mysqldbcopy utility's help text. (Bug #18206923, Bug #71627)
With Fabric, group definitions now use a single (global) username and password, instead of requiring a username/password for each MySQL server when they were added to the group. (Bug #18153823, Bug #71512)
When checking for proper permissions, Fabric would execute several SELECT statements against the 'mysql' database, which caused the login mechanism to fail if the user lacked access. This mechanism now uses 'SHOW GRANTS' statements instead, and now prints out a detailed error message when access is unavailable. (Bug #18138545, Bug #71448)
The server.set_status() function's status parameter now accepts an integer, to better coincide with the integer value provided by dump_servers(). The strings "PRIMARY", "SECONDARY", "SPARE" and "FAULTY" also remain as acceptable values. (Bug #18124108, Bug #71428)
The mysqlrpladmin utility did not correctly account for consecutive transactions reported in a single string. The utility would only consider the transaction or transaction range that appeared after the last ':'. (Bug #18083550)
The Installer (setup.py) was not able to install the fabric configuration file (fabric.cfg) to the user's home directory even though the --user or --home options were used. It would instead attempt to install it in /etc/mysql/. (Bug #18109179, Bug #71442)
On Linux, the mysqlserverclone utility failed to spawn a MySQL server if the path passed to the --new-data option was too long, as determined by the MySQL server. A check was added that limits the datadir length to 200 characters, which helps coincide with the maximum socket length (107) set by the MySQL server. This check can be bypassed with the --force option. (Bug #18117930)
The text for the store.dump_* commands was updated to better reflect how these commands are for retrieving information, instead of writing it. (Bug #18087356, Bug #71370)
When cloning a database with foreign key dependencies on the same MySQL server, and while using the multiprocessing option (with a value greater than one), the mysqldbcopy utility issued an error about foreign key constraint failures. The foreign key checks are now disabled before cloning the tables. (Bug #18041743)
The internal gtid_enabled() function would only check a single MySQL server, and returned TRUE if the checked MySQL server had GTID enabled. It now only returns TRUE if all MySQL servers in the topology have GTIDs enabled. (Bug #17890455)
The privilege check for the mysqlfailover utility was performed after the instance registration process, which would lead to query errors (instead of privilege errors) if some privileges, such as DROP or CREATE, were not available.
Additionally, a check for the INSERT privilege was added to all slaves, as it might be required for them to become the new master, in order to successfully perform the failover instance registration on the new master. (Bug #17878705)
The default TCP port set for MySQL Fabric was changed from 8080 to 32274. (Bug #17820905, Bug #70954)
The is_connected() fabric function was executed every time a statement was executed, which reduced overall performance. (Bug #17702237)
Exported or copied/cloned databases from the mysqldbcopy and mysqldbexport utilities would fail to function with the mysqldbimport utility if dependencies between views were present. These dependencies are now sorted in the appropriate order before copy, clone, or export. (Bug #17694651)
Some MUT tests were failing with Connector/Python 1.1.x due to format changes in the returned error messages by the Connector. (Bug #17655980)
The commercial version of the MySQL Utilities contained a hard-coded path to the Python directory, so the MySQL Utilities installation would fail. The Fabric and Doctrine projects were also updated. (Bug #17647498)
The mysqldiff and mysqldbcompare utilities print all of the control lines for each difference, even for unnecessary lines which is especially true of the grid format and unified types. A new --compact option was added, which reduces output from the diff. (Bug #14725008)
When executing the mysqldiskusage utility on a remote MySQL server, the utility would report information about the local datadir if the remote and local datadir paths were identical. (Bug #13572814)
The --force option was renamed to --drop-first for the mysqldbcopy utility. (Bug #13563798)
When using the mysqldbimport utility without the --drop-first option, importing a valid SQL import file that included a CREATE DATABASE statement for a database that already existed did not yield a related error message. The associated error message was updated, and it also refers to the optional --drop-first option. (Bug #13563798)
A new --use-indexes option was added to the mysqldbcompare utility. Before, this utility did not compare tables that lacked a primary key. The new --use-indexes option allows non-unique indexes to be used, although only indexes with columns not allowing null values are permitted.
Use --use-indexes to specify the unique index to be used for the Table comparative. If the specified index allows null values, then a warning is generated, and the next unique index will be used if one exists as otherwise the utility will stop with an error. (Bug #13492195)

New in MySQL Utilities 1.4.1 Alpha (Dec 18, 2013)

Functionality Added or Changed:
A new --license parameter was added to display the license text, and the --version parameter was updated to show the license type. (Bug #17634676)
An Installer package (DMG) was added for Mac OS X.
A workaround was to compile the MySQL Utilities from source. (Bug #17347424, Bug #70123)
The mysqlfrm utility now has the ability to produce a valid .frm file with the storage engine changed for any table that can be read by the .frm Reader in default mode. This ability requires you to specify --new-storage-engine and --frmdir (used to create and place .frm files, and the directory must exist). (Bug #17242369)
Performance improvements, including multiprocessing support, was added to the mysqldbexport, mysqldbimport and mysqldbcopy utilities. Performance improvements include:
mysqldbexport:
Multiprocessing by database for non-POSIX systems (Microsoft Windows) and by table for POSIX systems, using the new --multiprocess option.
A new --output-file parameter was added to store the export results instead of using the (slower) standard output. Export results are now always sent to a file, and only to stdout and the end if required.
mysqldbimport:
Multiprocessing by file (independent of the operating system), using the new --multiprocess option.
Perform only a single COMMIT at the end of importing each file, by default. A new --autocommit option was added to allow each statement to be committed upon its execution.
The bulk insert feature was improved (supports SQL statements). If the --bulk-insert option is enabled, then all INSERT statements are parsed and their data aggregated (if possible) in order to be executed as a single query.
A new --max-bulk-insert option was added to control the maximum size of the bulk insert, in order to avoid broken pipe errors. When the maximum number of INSERT statements is reached, then a new bulk insert is created.
mysqldbcopy:
Multiprocessing by database for non POSIX systems (Microsoft Windows) and by table for POSIX systems, using the new --multiprocess option.
Remove/disable the previous thread feature.
Bugs Fixed:
Checks are now made to force that a MySQL server belongs to a single group. (Bug #17929077, Bug #71125)
The mysqlmetagrep utility now emits an informative notice if an unsupported object type is passed in, such as "--search-objects=function" or "--object-types=function", when before it threw an unhandled exception. (Bug #17908146)
The required version of Connector/Python is now v1.0.8 or greater. (Bug #17864266)
Attempting to use the mysqldbcopy utility to copy a database with one or more blob fields would fail, and emit an error such as "TypeError: not all arguments converted during string formatting". (Bug #17722274)
On Microsoft Windows, the mysqlfabric script would not execute easily due to it missing a file extension. The file is now named mysqlfabric.py.
A workaround was to pass the script to a python executable, such as python scripts\mysqlfabric. (Bug #17639666, Bug #70694)
With Fabric, the checkpoint routine could cause deadlocks. (Bug #17633546, Bug #71127)
The help for the --rpl mysqldbexport utility parameter was improved to clarify that a destination/slave server cannot be specified for export. (Bug #17633465)
The mysqldbcopy and mysqldbexport utilities were creating views before user-defined functions or procedures. As result, the copy or export would fail if at least one of the views was using user-defined functions or procedures. Now, the functions and procedures are processed before the views. (Bug #17622298, Bug #70648)
On Microsoft Windows, starting Fabric could hang due to insufficient resources when used with a 32-bit build of Python on a 64-bit Windows system. Error handling was also improved for related use cases. (Bug #17592301, Bug #71128)
On Microsoft Windows, attempting to install the MySQL Utilities from source would fail, as setup.py required a "/etc" directory. (Bug #17555531, Bug #70512)
Improvements were made to the mysqldbcopy MUT tests. (Bug #17548335)
Error reporting in the mysqluc utility was improved to clarify which utilities are generating the errors. (Bug #17510350)
The mysqlauditadmin utility would unnecessarily set "audit_log_flush = ON" during a rotate operation. (Bug #17475780)
The mysqldbcopy utility could emit an error and exit when cloning a database with foreign key constraints that pointed to multiple tables. (Bug #17474810)
On Microsoft Windows, the list-commands parameter of the mysqlfabric utility did not function. (Bug #17454423, Bug #71126)
The mysqlindexcheck utility now informs the user when there was not enough data collected to produce a list of "best" and "worst" queries, when before it would seemingly ignore the table. (Bug #17457402, Bug #70339)
In MUT, the 'drop database' functionality is now centralized and reused throughout MUT, when before it was defined in multiple files. (Bug #17423074)
On Microsoft Windows, the mysqluc utility would crash after clicking the home button in MySQL Workbench. (Bug #17415167)
The Debian MySQL Utilities packages could not be installed if the installed Connector/Python package had a different license type, as the license type of both packages had to be identical. It is now possible to install a commercial version of Connector/Python with a GPL version of MySQL Utilities, or vice-versa. (Bug #17393742)
When MySQL Utilities was installed using the RPM package, it was not possible to upgrade with a newer version, or change the license type from another RPM package to another without manually removing the installed package. (Bug #17393523)
After executing the mysqluserclone utility without the optional --source parameter, which then attempted to use the default credentials, an unhandled exception would be thrown if the default credentials were invalid. The --source parameter is now required. (Bug #17217461)
The login-path feature, as set in .mylogin.cnf, did not allow for login paths with special characters, such as dashes. (Bug #17214291)
When the mysqlfailover connection to the master was killed, occasionally it caused an automatic fail over. mysqlfailover will now attempt to reconnect to the master, and after 3 attempts it will assume that the master is down. The connection to the master is not maintained between the reporting intervals. (Bug #17066910, Bug #69213)
The mysqlserverinfo utility parsed error messages raised during the server connection, which could suggest actions that the user could take to fix the errors, such as using the start option when the server was offline. The error numbers are now used for the same purpose instead of parsing the error messages. (Bug #16386941)
The server_info_errors test was unable to execute successfully on Microsoft Windows, and was temporary disabled. It is now enabled again, and was adjusted to function properly. (Bug #16226348)
The mysqlserverinfo utility output did not display information about the error, general, slow query. and audit log files. (Bug #14181681)
The mysqlserverclone utility did not check if the server passed to the --server parameter was local to the machine that mysqlserverclone was being executed on. (Bug #13773247)
The mysqlprocgrep utility did not provide an option to search and kill processes by ID. The --match-id parameter was added, which searches processes by ID and allows them to be killed by ID. (Bug #13572964)
The --body parameter of the mysqlmetagrep utility now supports views. (Bug #13563921)
The --character-set option was added to utilities that allow the user to define the client character set used for the MySQL connection, such as mysqldbexport and mysqldbimport. If this option is not provided, then the character set retrieved from the MySQL Server variable character_set_client is used. (Bug #13417229)