Chapter 17 MySQL Cluster NDB 6.1 - 7.1

Distribution of MySQL users and privileges.  Automatic distribution of MySQL users and privileges across all SQL nodes in a given MySQL Cluster is now supported. To enable this support, you must first import an SQL script share/mysql/ndb_dist_priv.sql that is included with the MySQL Cluster NDB 7.2 distribution. This script creates several stored procedures which you can use to enable privilege distribution and perform related tasks.

When a new MySQL Server joins a MySQL Cluster where privilege distribution is in effect, it also participates in the privilege distribution automatically.

Once privilege distribution is enabled, all changes to the grant tables made on any mysqld attached to the cluster are immediately available on any other attached MySQL Servers. This is true whether the changes are made using CREATE USER, GRANT, or any of the other statements described elsewhere in this Manual (see Section 13.7.1, “Account Management Statements”.) This includes privileges relating to stored routines and views; however, automatic distribution of the views or stored routines themselves is not currently supported.

For more information, see Section 17.5.14, “Distributed MySQL Privileges for MySQL Cluster”.

Distributed pushed-down joins.  Many joins can now be pushed down to the NDB kernel for processing on MySQL Cluster data nodes. Previously, a join was handled in MySQL Cluster by means of repeated accesses of NDB by the SQL node; however, when pushed-down joins are enabled, a pushable join is sent in its entirety to the data nodes, where it can be distributed among the data nodes and executed in parallel on multiple copies of the data, with a single, merged result being returned to mysqld. This can reduce greatly the number of round trips between an SQL node and the data nodes required to handle such a join, leading to greatly improved performance of join processing.

It is possible to determine when joins can be pushed down to the data nodes by examining the join with EXPLAIN. A number of new system status variables (Ndb_pushed_queries_defined, Ndb_pushed_queries_dropped, Ndb_pushed_queries_executed, and Ndb_pushed_reads) and additions to the counters table (in the ndbinfo information database) can also be helpful in determining when and how well joins are being pushed down.

More information and examples are available in the description of the ndb_join_pushdown server system variable. See also the description of the status variables referenced in the previous paragraph, as well as Section 17.5.10.7, “The ndbinfo counters Table”.

Improved default values for data node configuration parameters.  In order to provide more resiliency to environmental issues and better handling of some potential failure scenarios, and to perform more reliably with increases in memory and other resource requirements brought about by recent improvements in join handling by NDB, the default values for a number of MySQL Cluster data node configuration parameters have been changed. The parameters and changes are described in the following list:

In addition, the value computed for MaxNoOfLocalScans when this parameter is not set in config.ini has been increased by a factor of 4.

MySQL Cluster information database (ndbinfo).  The ndbinfo information database makes it possible to obtain real-time characteristics of a MySQL Cluster by issuing queries from the mysql client or other MySQL client applications. ndbinfo provides metadata specific to MySQL Cluster similarly to how the INFORMATION_SCHEMA database provides metadata for the standard MySQL Server. This eliminates much of the need to read log files, issue REPORT or DUMP commands in the ndb_mgm client, or parse the output of ndb_config in order to get configuration and status information from a running MySQL Cluster.

For more information, see Section 17.5.10, “The ndbinfo MySQL Cluster Information Database”.

Java connectors for MySQL Cluster.  The MySQL Cluster distribution now includes 2 new Java user APIs, ClusterJ and ClusterJPA. ClusterJ is an object-relational interface in a manner similar to that of Java persistence frameworks such as Hibernate. Cluster JPA is a reimplementation of OpenJPA. ClusterJ uses a backend library (NdbJTie) that provides access to the NDB storage engine without using a MySQL Server connection or JDBC. ClusterJPA also uses NdbJTie when it improves performance, but can also process complex queries using JDBC and a MySQL Server connection, where it can take advantage of the MySQL query optimizer.

ClusterJ and Cluster JPA can also be made to work with recent MySQL Cluster NDB 7.0 releases although the necessary library and JAR files are included only in MySQL Cluster NDB 7.1.1 and later.

For more information about using ClusterJ and ClusterJPA, see MySQL Cluster Connector for Java.

New CHANGE MASTER TO option for circular replication.  Beginning with MySQL Cluster NDB 7.1.0, the CHANGE MASTER TO statement supports an IGNORE_SERVER_IDS option which takes a comma-separated list of server IDs and causes events originating from the corresponding servers to be ignored. (Log rotation and log deletion events are preserved.)

See Section 13.4.2.1, “CHANGE MASTER TO Syntax”, as well as Section 13.7.5.36, “SHOW SLAVE STATUS Syntax”, for more information.

Native support for default column values.  Starting with MySQL Cluster NDB 7.1.4, default values for table columns are stored in the NDB kernel rather than by the MySQL server as was done previously. This means that inserts on tables having column value defaults can be smaller and faster than before, because less data must be sent from SQL nodes to NDBCLUSTER.

Tables created using previous MySQL Cluster releases can still be used in MySQL Cluster 7.1.4 and later; however, they do not support native default values until they are upgraded. You can upgrade a table with non-native default values to support native default values using an offline ALTER TABLE statement.

MySQL Cluster on Windows (Production).  Beginning with MySQL Cluster NDB 7.1.3, MySQL Cluster is available for production use on Microsoft Windows operating systems; MySQL Cluster NDB 7.1 binaries for Windows can be obtained from cluster/.

Features and behavior are generally comparable to those found on previously supported platforms such as Linux and Solaris. However, you must install the binaries manually.

Beginning with MySQL Cluster NDB 7.1.5, MySQL Cluster processes can be run as Windows services.

If you wish to build MySQL Cluster from source on Windows, you must configure the build using the WITH_NDBCLUSTER_STORAGE_ENGINE option. For more information, see Section 2.11.7, “Installing MySQL from Source on Windows”.

--nowait-nodes option for management servers.  It is now possible to configure a cluster with two management servers, but to start the cluster using only one of them by starting the management node daemon with the --nowait-nodes option. The other management server can then be started at a later time to join the running MySQL Cluster.

Improved lock handling for primary key lookups on BLOB tables.  A MySQL Cluster table stores all but the first 256 bytes of any BLOB or TEXT column values in a separate BLOB table; when executing queries against such tables, a shared lock is obtained. Prior to MySQL Cluster NDB 7.1.1, when the query used a primary key lookup and took place within a transaction, the lock was held for the duration of the transaction, even after no more data was being read from the NDB table. Now in such cases, the lock is released when all BLOB data associated with the table has been read. (Bug #49190)

Heartbeat thread policy and priority.  Beginning with MySQL Cluster NDB 7.1.2, a new configuration parameter HeartbeatThreadPriority makes it possible to set the policy and the priority for the heartbeat thread on management and API nodes.

Improved access to partitioning information.  The ndb_desc utility now provides additional information about the partitioning of data stored in MySQL Cluster. Beginning with MySQL Cluster NDB 7.1.2, the --blob-info option causes this program to include partition information for BLOB tables in its output. Also beginning with MySQL Cluster NDB 7.1.2, the --extra-node-info option causes ndb_desc to include information about data distribution (that is, which table fragments are stored on which data nodes). Each of these options also requires the use of the --extra-partition-info option.

Information about partition-to-node mappings can also be obtained using the Table::getFragmentNodes() method, also added in MySQL Cluster NDB 7.1.2.

Replication attribute promotion and demotion.  Beginning with MySQL Cluster NDB 7.1.3, MySQL Cluster Replication supports attribute promotion and demotion when replicating between columns of different but similar types on the master and the slave. For example, it is possible to promote an INT column on the master to a BIGINT column on the slave, and to demote a TEXT column to a VARCHAR column.

The implementation of type demotion distinguishes between lossy and non-lossy type conversions, and their use on the slave can be controlled by setting the slave_type_conversions global server system variable.

For more information, see Attribute promotion and demotion (MySQL Cluster).

Change in ndbinfo database.  The experimental pools table was removed from ndbinfo in MySQL Cluster NDB 7.1.3. Applications which used this table can and should be rewritten to use other ndbinfo tables.

Configuration caching control.  Beginning with MySQL Cluster NDB 7.1.4, it is possible to disable the management server's configuration cache using the --config-cache option, which forces ndb_mgmd to read its configuration data from the config.ini configuration file every time it starts. For more information about configuration caching and this option, see Section 17.3.2, “MySQL Cluster Configuration Files”. See also Section 17.4.4, “ndb_mgmd — The MySQL Cluster Management Server Daemon”.

Incompatible change in NDB API event reporting.  Beginning with MySQL Cluster NDB 7.1.4, DDL events are no longer reported on Event objects by default. Instead the desired level of event reporting (EventReport value) must be set explicitly using Event::setReport().

Number of table attributes.  Beginning with MySQL Cluster NDB 7.1.4, the maximum number of attributes (columns plus indexes) per table has been increased from 128 to 512.

InnoDB support in commercial binaries.  Beginning with MySQL Cluster NDB 7.1.4b, all commercial binary releases of MySQL Cluster provide support for the InnoDB storage engine.

Heartbeat ordering.  Beginning with MySQL Cluster NDB 7.1.5, it is possible to set a specific order for transmission of heartbeats between data nodes, using the HeartbeatOrder data node configuration parameter introduced in this version. This parameter can be useful in situations where multiple data nodes are running on the same host and a temporary disruption in connectivity between hosts would otherwise cause the loss of a node group (and thus failure of the cluster).

Relaxed ndb_restore column comparison rules.  When restoring data, ndb_restore compares the attributes of a column for equality with the definition of the column in the target table. However, not all of these attributes need to be the same for ndb_restore to be meaningful, safe and useful. Beginning with MySQL Cluster NDB 7.1.5, ndb_restore automatically ignores differences in certain column attributes which do not necessarily have to match between the version of the column in a backup and the version of that column in the MySQL Cluster to which the column data is being restored. These attributes include the following:

In such cases, ndb_restore reports any such differences to minimize any chance of user error.

Storage of user data in anyValue.  When writing NDB events to the binary log, MySQL Cluster uses OperationOptions::anyValue to store the server ID. Beginning with MySQL Cluster NDB 7.1.6, it is possible to store user data from an NDB API application in part of the anyValue when mysqld has been started with the --server-id-bits option set to a nondefault value. Also beginning with MySQL Cluster NDB 7.1.6, it is possible to view this data in the output of mysqlbinlog, for which its own --server-id-bits option is added.

--add-drop-trigger option for mysqldump.  Beginning with MySQL Cluster NDB 7.1.8, this option can be used to force all CREATE TRIGGER statements in mysqldump output to be preceded by a DROP TRIGGER IF EXISTS statement.

Forcing node shutdown and restart.  In MySQL Cluster NDB 7.1.8 and later, it is possible using the ndb_mgm management client or the MGM API to force a data node shutdown or restart even if this would force the shutdown or restart of the entire cluster. In the management client, this is implemented through the addition of the -f (force) option to the STOP and RESTART commands. For more information, see Section 17.5.2, “Commands in the MySQL Cluster Management Client”. The MGM API also adds two new methods for forcing such a node shutdown or restart; see ndb_mgm_stop4(), and ndb_mgm_restart4(), for more information about these methods.

Disk Data usage statistics (diskpagebuffer table).  MySQL Cluster 7.1.9 introduces a new table in the ndbinfo information database. The diskpagebuffer table provides real-time data on disk page buffer usage. These statistics can be used to monitor performance of MySQL Cluster Disk Data read and write operations, and can prove useful in the tuning of Disk Data parameters such as DiskPageBufferMemory.

InnoDB Plugin support.  Beginning with MySQL Cluster NDB 7.1.9, the MySQL Server supplied with MySQL Cluster supports the InnoDB Plugin. For more information about enabling the plugin if you are building MySQL Cluster from source, see Section 17.2, “MySQL Cluster Installation and Upgrades”.

TimeBetweenEpochsTimeout and GCP stop control.  Beginning with MySQL Cluster NDB 7.1.10, it is possible to disable GCP stops by setting TimeBetweenEpochsTimeout to 0. In addition, a warning is written to the cluster log whenever the time required for a GCP save exceeds 60 seconds or the time required for a GCP commit exceeds 10 seconds. This warning includes a report of the current value of TimeBetweenEpochsTimeout. For more information, see Disk Data and GCP Stop errors.

Skipping corrupted tables in NDB native backups.  Beginning with MySQL Cluster NDB 7.1.10, you can cause ndb_restore to ignore tables that are corrupted due to missing blob parts tables by using the --skip-broken-objects option. When this option is used, such tables are skipped, and the restoration of any remaining uncorrupted tables in the backup continues.

BLOB read and write batching.  Beginning with MySQL Cluster NDB 7.1.10, it possible to control batching of BLOB read and write operations. For SQL nodes, this can be done using the --ndb-blob-read-batch-bytes and --ndb-blob-write-batch-bytes options for mysqld. In NDB API applications, you can control batching of BLOB reads and writes using the NdbTransaction methods setMaxPendingBlobReadBytes(), getMaxPendingBlobReadBytes(), setMaxPendingBlobWriteBytes(), and getMaxPendingBlobWriteBytes().

Restoring from a NDB native backup to a differently-named database.  MySQL Cluster NDB 7.1.11 adds a new --rewrite-database option to ndb_restore, which makes it possible to restore to a database having a different name from that of the database in the backup. The option can be used multiple times, and it is possible to restore from more than one source database in the backup to a single target database (although no protection against table or other object name collision is provided).

See Section 17.4.20, “ndb_restore — Restore a MySQL Cluster Backup”, for more information.

Selective over-commit handling.  Beginning with MySQL Cluster NDB 7.1.10, it is possible to exercise more direct control over uncommitted operations from transactions aborted due to timeouts flushing redo logs to disk. This is implemented using three configuration parameters added in this version of MySQL Cluster: the data node configuration parameters RedoOverCommitCounter and RedoOverCommitLimit, and the API node configuration parameter DefaultOperationRedoProblemAction.

When an attempt to flush a given redo log takes longer than RedoOverCommitLimit seconds, and this occurs RedoOverCommitLimit times or more, the transactions contained within the redo log are aborted. Any operations left uncommitted as a result are either aborted or re-tried, according to the value of DefaultOperationRedoProblemAction.

For more information, see Redo log over-commit handling

INFORMATION_SCHEMA improvements.  Beginning with MySQL Cluster NDB 7.1.11, Beginning with MySQL Cluster NDB 7.0.22, INFORMATION_SCHEMA provides disk usage information for MySQL Cluster Disk Data tables. Previously, INFORMATION_SCHEMA.TABLES showed only the space usage for the in-memory data part of the table. Now, it also shows the space allocated for and used by the disk_part data of that table as well.

In addition, the INFORMATION_SCHEMA.PARTITIONS table (which did not show any statistics for NDB tables) now shows correct values in this table's TABLE_ROWS, AVG_ROW_LENGTH, DATA_LENGTH, MAX_DATA_LENGTH, and DATA_FREE columns, for each partition.

ndb_restore attribute demotion.  Beginning with MySQL Cluster NDB 7.1.11, it is possible to enable attribute demotion when restoring a MySQL Cluster from a native backup running ndb_restore with a new --lossy-conversions option.

In general,the rules governing demotion are the same as for MySQL replication, although there are some exceptions that you may need to take into account being employing this option. See Section 16.4.1.9.2, “Replication of Columns Having Different Data Types”, for information about type conversions currently supported by attribute promotion and demotion in MySQL Cluster.

Improved multi-threaded order index building.  Previously, it was not possible to enable multi-threaded building of ordered indexes during initial restarts. In MySQL Cluster NDB 7.1.11, this can now be done, using the new TwoPassInitialNodeRestartCopy data node configuration parameter.

Configuration version information in ndbinfo.nodes.  You can see which version or versions of the MySQL Cluster configuration file are in effect on the data nodes by checking the config_generation column which is added to the nodes table in MySQL Cluster NDB 7.1.13.

Configuration version information in ndbinfo.nodes.  You can see which version or versions of the MySQL Cluster configuration file are in effect on the data nodes by checking the config_generation column which is added to the nodes table in MySQL Cluster NDB 7.1.13.

Improvements in adding data nodes online.  Begining with MySQL Cluster NDB 7.0.24, it is possible to add data nodes online to a running MySQL Cluster without performing a rolling restart of the cluster or starting data node processes with the --nowait-nodes option. This can be done by setting Nodegroup = 65536 in the config.ini file for any data nodes that should be started at a later time, when first starting the cluster. The amount of time the cluster waits before doing this can be controlled using the StartNoNodeGroupTimeout data node configuration parameter.

Unique key updates in replication.  It is possible in MySQL Cluster NDB 7.1.14 and later to employ operations that update unique keys when replicating NDB tables. Previously this could lead to duplicate key errors when trying to execute the binary log (due to the fact that row events in the binary log were ordered according to the partitioning of the base table, and could differ in order within the epoch for that in which they were executed).

Starting with MySQL Cluster NDB 7.1.17, the NDB kernel implements a number of statistical counters relating to actions performed by or affecting Ndb objects. Such events include starting, closing, and aborting transactions; operations using primary keys or unique keys; table, range, and pruned scans; blocking of threads by incompleted operations; and data and events sent and received by MySQL Cluster nodes. NDB API statistics counters are incremented inside the NDB kernel whenever NDB API calls are made or data is sent to or received by the data nodes. A MySQL Server running as an SQL node in the cluster can access the values of these counters as system status variables, as seen in the output of SHOW STATUS, or in the results of queries against the SESSION_STATUS or GLOBAL_STATUS table in the INFORMATION_SCHEMA database. By comparing the values of these status variables before and after the execution of statements affecting NDB tables, you can observe the corresponding actions taken on the NDB API level. This can be highly useful when monitoring and tuning MySQL Cluster.

For more information, see Section 17.5.15, “NDB API Statistics Counters and Variables”. See also Section 17.3.4.4, “MySQL Cluster Status Variables”, for information about the individual status variables.

Version 2 binary log row events.  New versions of the WRITE_ROW, UPDATE_ROW, and DELETE_ROW events have been implemented in MySQL Cluster NDB 7.1.16, extending them with additional information intended to support future enhancements; these are referred to as “Version 2” binary log events.

Version 2 log events are not backward compatible, and cannot be read by older slaves. A new mysqld option --log-bin-use-v1-row-events can be employed to force use of Version 1 events when writing the binary log. This can be used during upgrades to make a newer mysqld generate Version 1 binary log row events that can be read by older slaves.

Circular replication: Primary wins first conflict detection.  Two new conflict detection functions NDB$EPOCH() and NDB$EPOCH_TRANS() can be useful in “active-active” circular replication scenarios with two MySQL Clusters. For each case, we designate one cluster as primary and one as secondary, and implement a “primary always wins” rule for determining whether to accept conflicting changes. When using NDB$EPOCH(), conflicting rows on the secondary are realigned with those on the primary; when using NDB$EPOCH_TRANS(), it is transactions containing rows in conflict (and any transactions which depend on them) on the secondary that are realigned.

When using NDB$EPOCH_TRANS() as the conflict detection function, the binary log must be written using Version 2 binary logging row events; that is, the mysqld processes on both the primary and the secondary must be started with --log-bin-use-v1-row-events=0. In addition, the secondary's binary log must include transaction IDs for all rows, that is, by setting --ndb-log-transaction-id=1.

You can monitor conflict detection and resolution performed using these functions by reading a number of related server status variables, including Ndb_conflict_fn_epoch, Ndb_conflict_fn_epoch_trans, and Ndb_conflict_trans_row_conflict_count (among others). See Section 17.3.4.4, “MySQL Cluster Status Variables”.

For more information, see Section 17.6.11, “MySQL Cluster Replication Conflict Resolution”.

Fail-fast data node capability.  Beginning with MySQL Cluster NDB 7.1.17, it is possible to enable fail-fast behavior for data nodes by enabling the CrashOnCorruptedTuple configuration parameter introduced in this version (disabled by default). Doing so causes data nodes to fail whenever they detect a corrupted tuple.

Rows per partition limit removed.  Previously it was possible to store a maximum of 46137488 rows in a single MySQL Cluster partition—that is, per data node. Beginning with MySQL Cluster NDB 7.0.36 and MySQL Cluster NDB 7.1.25, this limitation has been lifted, and there is no longer any practical upper limit to this number. (Bug #13844405, Bug #14000373)

MySQL Cluster on Windows (alpha).  MySQL Cluster NDB 7.0 is available on an experimental basis for Windows operating systems (for production use on Windows, you should use MySQL Cluster NDB 7.1.3 or later). Features and behavior comparable to those found on platforms that are already supported—such as Linux and Solaris—are planned for MySQL Cluster on Windows. In MySQL Cluster NDB 7.0, you must build from source (Windows binaries are available for MySQL Cluster NDB 7.1 releases). To enable MySQL Cluster support on Windows when building from source, you must configure the build using the WITH_NDBCLUSTER_STORAGE_ENGINE option. For more information, see Section 2.11.7, “Installing MySQL from Source on Windows”.

Ability to add nodes and node groups online.  Beginning with MySQL Cluster NDB 6.4.0, it is possible to add new node groups (and thus new data nodes) to a running MySQL Cluster without shutting down and reloading the cluster. As part of enabling this feature, a new command CREATE NODEGROUP has been added to the cluster management client and the functionality of the ALTER ONLINE TABLE ... REORGANIZE PARTITION SQL statement has been extended. For more information, see Section 17.5.13, “Adding MySQL Cluster Data Nodes Online”.

Data node multi-threading support.  Beginning with MySQL Cluster NDB 6.4.0, a multi-threaded version of the data node daemon, named ndbmtd, is available for use on data node hosts with multiple CPU cores. This binary is built automatically when compiling with MySQL Cluster support; no additional options other than those needed to provide MySQL Cluster support are needed when configuring the build. In most respects, ndbmtd functions in the same way as ndbd, and can use the same command-line options and configuration parameters. In addition, the new MaxNoOfExecutionThreads configuration parameter can be used to determine the number of data node process threads for ndbmtd. For more information, see Section 17.4.3, “ndbmtd — The MySQL Cluster Data Node Daemon (Multi-Threaded)”.

Configuration cache.  Formerly, MySQL Cluster configuration was stateless—that is, configuration information was reloaded from the cluster's global configuration file (usually config.ini) each time ndb_mgmd was started. Beginning with MySQL Cluster NDB 6.4.0, the cluster's configuration is cached internally, and the global configuration file is no longer automatically re-read when the management server is restarted. This behavior can be controlled using the management server options --configdir, --initial, and --reload. In MySQL Cluster NDB 7.0.15 and later, the configuration cache can be disabled using the --config-cache option. For more information about these changes, see Section 17.3.2, “MySQL Cluster Configuration Files”. For more information about the new management server options, see Section 17.4.4, “ndb_mgmd — The MySQL Cluster Management Server Daemon”.

Detection of NDB API client connection errors.  In MySQL Cluster NDB 7.0 (6.4.0 and later releases), the NDB API's Ndb_cluster_connection class adds the get_latest_error() and get_latest_error_msg() methods for catching and diagnosing problems with NDB API client connections.

Snapshot options for backups.  Beginning with MySQL Cluster NDB 6.4.0, you can determine when performing a cluster backup whether the backup matches the state of the data when the backup was started or when it was completed, using the new options SNAPSHOTSTART and SNAPSHOTEND for the management client's START BACKUP command. See Section 17.5.3.2, “Using The MySQL Cluster Management Client to Create a Backup”, for more information.

Dynamic NDB transporter send buffer memory allocation.  Previously, the NDB kernel used a fixed-size send buffer for every data node in the cluster, which was allocated when the node started. Because the size of this buffer could not be changed after the cluster was started, it was necessary to make it large enough in advance to accommodate the maximum possible load on any transporter socket. However, this was an inefficient use of memory, since much of it often went unused. Beginning with MySQL Cluster NDB 6.4.0, send buffer memory is allocated dynamically from a memory pool shared between all transporters, which means that the size of the send buffer can be adjusted as necessary. This change is reflected by the addition of the configuration parameters TotalSendBufferMemory, ReservedSendBufferMemory, and OverLoadLimit, as well as a change in how the existing SendBufferMemory configuration parameter is used. For more information, see Section 17.3.2.13, “Configuring MySQL Cluster Send Buffer Parameters”.

Robust DDL operations.  Beginning with MySQL Cluster NDB 6.4.0, DDL operations (such as CREATE TABLE or ALTER TABLE) are protected from data node failures; in the event of a data node failure, such operations are now rolled back gracefully. Previously, if a data node failed while trying to perform a DDL operation, the MySQL Cluster data dictionary became locked and no further DDL statements could be executed without restarting the cluster.

IPv6 support in MySQL Cluster Replication.  Beginning with MySQL Cluster NDB 6.4.1, IPv6 networking is supported between MySQL Cluster SQL nodes, which makes it possible to replicate between instances of MySQL Cluster using IPv6 addresses. However, IPv6 is supported only for direct connections between MySQL servers; all connections within an individual MySQL Cluster must use IPv4. For more information, see Section 17.6.3, “Known Issues in MySQL Cluster Replication”.

Restoring specific databases, tables, or columns from a MySQL Cluster backup.  It is now possible to exercise more fine-grained control when restoring a MySQL Cluster from backup using ndb_restore. Beginning with MySQL Cluster NDB 6.4.3, you can choose to restore only specified tables or databases, or exclude specific tables or databases from being restored, using the new ndb_restore options --include-tables, --include-databases, --exclude-tables, and --exclude-databases. Beginning with MySQL Cluster NDB 7.0.7, it is also possible to restore to a table having fewer columns than the original using the --exclude-missing-columns option. For more information about all of these options, see Section 17.4.20, “ndb_restore — Restore a MySQL Cluster Backup”.

Improved Disk Data file system configuration.  As of MySQL Cluster NDB 6.4.3, you can specify default locations for MySQL Cluster Disk Data data files and undo log files using the data node configuration parameters FileSystemPathDD, FileSystemPathDataFiles, and FileSystemPathUndoFiles. This eliminates the need to use symbolic links to place Disk Data files separately from other files in data node file systems to improve Disk Data performance. For more information, see Disk Data file system parameters.

Automatic creation of Disk Data log file groups and tablespaces.  Beginning with MySQL Cluster NDB 6.4.3, using the data node configuration parameters InitialLogFileGroup and InitialTablespace, you can cause the creation of a MySQL Cluster Disk Data log file group, tablespace, or both, when the cluster is first started. When using these parameters, no SQL statements are required to create these Disk Data objects. For more information, see Disk Data object creation parameters.

Improved internal message passing and record handling.  MySQL Cluster NDB 7.0 contains 2 changes that optimize the use of network connections by addressing the size and number of messages passed between data nodes, and between data nodes and API nodes, which can increase MySQL Cluster and application performance:

Both of these optimizations are internal to the NDB API, and so is transparent to applications; this is true whether an application uses the NDB API directly or does so indirectly through an SQL node.

Configuration parameter data dumps.  Starting with MySQL Cluster NDB 7.0.6, the ndb_config utility supports a --configinfo option that causes it to dump a list of all configuration parameters supported by the cluster, along with brief descriptions, information about the parameters' default and permitted values, and the sections of the config.ini file in which the parameters apply. An additional --xml switch causes ndb_config to use XML rather than plaintext output. Using ndb_config --configinfo or ndb_config --configinfo --xml requires no access to a running MySQL Cluster, any other programs, or any files. For more information and examples, see Section 17.4.7, “ndb_config — Extract MySQL Cluster Configuration Information”.

Per-table reporting of free space on disk.  The INFORMATION_SCHEMA.FILES table shows information about used and free space in MySQL Cluster Disk Data data files, but this information is not applicable to individual tables. In MySQL Cluster NDB 7.0.8 and later, the ndb_desc utility provides two additional columns in its output that show the amount of space allocated on disk for a given NDB table as well the amount of space that remains available for additional storage of disk-based column data for that table. For more information, see Section 17.4.10, “ndb_desc — Describe NDB Tables”.

Improved restart times.  Optimizations in redo log handling and other file system operations introduced in MySQL Cluster NDB 7.0.9 have the potential to reduce considerably the time required for restarts. While actual performance benefits observed in production setups will naturally vary depending on database size, hardware, and other conditions, our own preliminary testing has shown that these improvements can yield startup times that are faster than those typical of previous MySQL Cluster NDB 7.0 releases by a factor of 50 or more.

Native support for default column values.  Starting with MySQL Cluster NDB 7.0.15, default values for table columns are stored in the NDB kernel rather than by the MySQL server as was done previously. This means that inserts on tables having column value defaults can be smaller and faster than before, because less data must be sent from SQL nodes to NDBCLUSTER.

Tables created using previous MySQL Cluster releases can still be used in MySQL Cluster 7.0.15 and later; however, they do not support native default values until they are upgraded. You can upgrade a table with non-native default values to support native default values using an offline ALTER TABLE statement.

--nowait-nodes option for management servers.  Starting with MySQL Cluster NDB 7.0.10, it is possible to configure a cluster with two management servers, but to start the cluster using only one of them by starting the management node daemon with the --nowait-nodes option. The other management server can then be started at a later time to join the running MySQL Cluster.

Increased flexibility in online upgrade procedure.  Previously, when performing an upgrade of a running MySQL cluster, the order in which the types of cluster nodes had to be upgraded was very strict. However, beginning with MySQL Cluster NDB 7.0.10, MySQL Cluster supports online upgrading of API nodes (including MySQL servers running as SQL nodes) online upgrading management nodes, data nodes, or both.

New CHANGE MASTER TO option for circular replication.  Beginning with MySQL Cluster NDB 7.0.11, the CHANGE MASTER TO statement supports an IGNORE_SERVER_IDS option which takes a comma-separated list of server IDs and causes events originating from the corresponding servers to be ignored. (Log rotation and log deletion events are preserved.)

See Section 13.4.2.1, “CHANGE MASTER TO Syntax”, as well as Section 13.7.5.36, “SHOW SLAVE STATUS Syntax”, for more information.

New replication conflict resolution strategy.  Beginning with MySQL Cluster NDB 7.0.11, the function NDB$MAX_DELETE_WIN() is available to implement “greatest timestamp, delete wins” conflict resolution. See NDB$MAX_DELETE_WIN(column_name), for more information.

Improved lock handling for primary key lookups on BLOB tables.  A MySQL Cluster table stores all but the first 256 bytes of any BLOB or TEXT column values in a separate BLOB table; when executing queries against such tables, a shared lock is obtained. Prior to MySQL Cluster NDB 7.0.12, when the query used a primary key lookup and took place within a transaction, the lock was held for the duration of the transaction, even after no more data was being read from the NDB table. Now in such cases, the lock is released when all BLOB data associated with the table has been read. (Bug #49190)

Heartbeat thread policy and priority.  Beginning with MySQL Cluster NDB 7.0.13, a new configuration parameter HeartbeatThreadPriority makes it possible to set the policy and the priority for the heartbeat thread on management and API nodes.

Improved access to partitioning information.  The ndb_desc utility now provides additional information about the partitioning of data stored in MySQL Cluster. Beginning with MySQL Cluster NDB 7.0.13, the --blob-info option causes this program to include partition information for BLOB tables in its output. Beginning with MySQL Cluster NDB 7.0.14, the --extra-node-info option causes ndb_desc to include information about data distribution (that is, which table fragments are stored on which data nodes). Each of these options also requires the use of the --extra-partition-info option.

Information about partition-to-node mappings can also be obtained using the Table::getFragmentNodes() method, also added in MySQL Cluster NDB 7.0.14.

Replication attribute promotion and demotion.  Beginning with MySQL Cluster NDB 7.0.14, MySQL Cluster Replication supports attribute promotion and demotion when replicating between columns of different but similar types on the master and the slave. For example, it is possible to promote an INT column on the master to a BIGINT column on the slave, and to demote a TEXT column to a VARCHAR column.

The implementation of type demotion distinguishes between lossy and non-lossy type conversions, and their use on the slave can be controlled by setting the slave_type_conversions global server system variable.

For more information, see Attribute promotion and demotion (MySQL Cluster).

Incompatible change in NDB API event reporting.  Beginning with MySQL Cluster NDB 7.0.15, DDL events are no longer reported on Event objects by default. Instead such event reporting must be enabled explicitly using the Event::setReport() method. For more information, see Event::setReport(), and The Event::EventReport Type.

Number of table attributes.  Beginning with MySQL Cluster NDB 7.0.15, the maximum number of attributes (columns plus indexes) per table has been increased from 128 to 512.

Heartbeat ordering.  Beginning with MySQL Cluster NDB 7.0.16, it is possible to set a specific order for transmission of heartbeats between data nodes, using the HeartbeatOrder data node configuration parameter introduced in this version. This parameter can be useful in situations where multiple data nodes are running on the same host and a temporary disruption in connectivity between hosts would otherwise cause the loss of a node group (and thus failure of the cluster).

Relaxed ndb_restore column comparison rules.  When restoring data, ndb_restore compares the attributes of a column for equality with the definition of the column in the target table. However, not all of these attributes need to be the same for ndb_restore to be meaningful, safe and useful. Beginning with MySQL Cluster NDB 7.0.16, ndb_restore automatically ignores differences in certain column attributes which do not necessarily have to match between the version of the column in a backup and the version of that column in the MySQL Cluster to which the column data is being restored. These attributes include the following:

In such cases, ndb_restore reports any such differences to minimize any chance of user error.

Storage of user data in anyValue.  When writing NDB events to the binary log, MySQL Cluster uses OperationOptions::anyValue to store the server ID. Beginning with MySQL Cluster NDB 7.0.17, it is possible to store user data from an NDB API application in part of the anyValue when mysqld has been started with the --server-id-bits option set to a nondefault value. Also beginning with MySQL Cluster NDB 7.0.17, it is possible to view this data in the output of mysqlbinlog, for which its own --server-id-bits option is added.

--add-drop-trigger option for mysqldump.  Beginning with MySQL Cluster NDB 7.0.19, this option can be used to force all CREATE TRIGGER statements in mysqldump output to be preceded by a DROP TRIGGER IF EXISTS statement.

Forcing node shutdown and restart.  In MySQL Cluster NDB 7.0.19 and later, it is possible using the ndb_mgm management client or the MGM API to force a data node shutdown or restart even if this would force the shutdown or restart of the entire cluster. In the management client, this is implemented through the addition of the -f (force) option to the STOP and RESTART commands. For more information, see Section 17.5.2, “Commands in the MySQL Cluster Management Client”. The MGM API also adds two new methods for forcing such a node shutdown or restart; see ndb_mgm_stop4(), and ndb_mgm_restart4(), for more information about these methods.

TimeBetweenEpochsTimeout and GCP stop control.  Beginning with MySQL Cluster NDB 7.0.21, it is possible to disable GCP stops by setting TimeBetweenEpochsTimeout to 0. In addition, a warning is written to the cluster log whenever the time required for a GCP save exceeds 60 seconds or the time required for a GCP commit exceeds 10 seconds. This warning includes a report of the current value of TimeBetweenEpochsTimeout. For more information, see Disk Data and GCP Stop errors.

Skipping corrupted tables in NDB native backups.  Beginning with MySQL Cluster NDB 7.0.21, you can cause ndb_restore to ignore tables that are corrupted due to missing blob parts tables by using the --skip-broken-objects option. When this option is used, such tables are skipped, and the restoration of any remaining uncorrupted tables in the backup continues.

BLOB read and write batching.  Beginning with MySQL Cluster NDB 7.0.21, it possible to control batching of BLOB read and write operations. For SQL nodes, this can be done using the --ndb-blob-read-batch-bytes and --ndb-blob-write-batch-bytes options for mysqld. In NDB API applications, you can control batching of BLOB reads and writes using the NdbTransaction methods setMaxPendingBlobReadBytes(), getMaxPendingBlobReadBytes(), setMaxPendingBlobWriteBytes(), and getMaxPendingBlobWriteBytes().

Restoring from a NDB native backup to a differently-named database.  MySQL Cluster NDB 7.0.22 adds a new --rewrite-database option to ndb_restore, which makes it possible to restore to a database having a different name from that of the database in the backup. The option can be used multiple times, and it is possible to restore from more than one source database in the backup to a single target database (although no protection against table or other object name collision is provided).

See Section 17.4.20, “ndb_restore — Restore a MySQL Cluster Backup”, for more information.

INFORMATION_SCHEMA inprovements.  Beginning with MySQL Cluster NDB 7.0.22, INFORMATION_SCHEMA provides disk usage information for MySQL Cluster Disk Data tables. Previously, INFORMATION_SCHEMA.TABLES showed only the space usage for the in-memory data part of the table. Now, it also shows the space allocated for and used by the disk_part data of that table as well.

In addition, the INFORMATION_SCHEMA.PARTITIONS table (which did not show any statistics for NDB tables) now shows correct values in this table's TABLE_ROWS, AVG_ROW_LENGTH, DATA_LENGTH, MAX_DATA_LENGTH, and DATA_FREE columns, for each partition.

Configuration version information in ndbinfo.nodes.  You can see which version or versions of the MySQL Cluster configuration file are in effect on the data nodes by checking the config_generation column which is added to the nodes table in MySQL Cluster NDB 7.0.24.

Improvements in adding data nodes online.  Beginning with MySQL Cluster NDB 7.0.24, it is possible to add data nodes online to a running MySQL Cluster without performing a rolling restart of the cluster or starting data node processes with the --nowait-nodes option. This can be done by setting Nodegroup = 65536 in the config.ini file for any data nodes that should be started at a later time, when first starting the cluster. The amount of time the cluster waits before doing this can be controlled using the StartNoNodeGroupTimeout data node configuration parameter.

Unique key updates in replication.  It is possible in MySQL Cluster NDB 7.0.25 and later to employ operations that update unique keys when replicating NDB tables. Previously this could lead to duplicate key errors when trying to execute the binary log (due to the fact that row events in the binary log were ordered according to the partitioning of the base table, and could differ in order within the epoch for that in which they were executed).

Starting with MySQL Cluster NDB 7.0.22, the NDB kernel implements a number of statistical counters relating to actions performed by or affecting Ndb objects. Such events include starting, closing, and aborting transactions; operations using primary keys or unique keys; table, range, and pruned scans; blocking of threads by incompleted operations; and data and events sent and received by MySQL Cluster nodes. NDB API statistics counters are incremented inside the NDB kernel whenever NDB API calls are made or data is sent to or received by the data nodes. A MySQL Server running as an SQL node in the cluster can access the values of these counters as system status variables, as seen in the output of SHOW STATUS, or in the results of queries against the SESSION_STATUS or GLOBAL_STATUS table in the INFORMATION_SCHEMA database. By comparing the values of these status variables before and after the execution of statements affecting NDB tables, you can observe the corresponding actions taken on the NDB API level. This can be highly useful when monitoring and tuning MySQL Cluster.

For more information, see Section 17.5.15, “NDB API Statistics Counters and Variables”. See also Section 17.3.4.4, “MySQL Cluster Status Variables”, for information about the individual status variables.

Version 2 binary log row events.  New versions of the WRITE_ROW, UPDATE_ROW, and DELETE_ROW events have been implemented in MySQL Cluster NDB 7.0.27, extending them with additional information intended to support future enhancements; these are referred to as “Version 2” binary log events.

Version 2 log events are not backward compatible, and cannot be read by older slaves. A new mysqld option --log-bin-use-v1-row-events can be employed to force use of Version 1 events when writing the binary log. This can be used during upgrades to make a newer mysqld generate Version 1 binary log row events that can be read by older slaves.

Fail-fast data node capability.  Beginning with MySQL Cluster NDB 7.0.28, it is possible to enable fail-fast behavior for data nodes by enabling the CrashOnCorruptedTuple configuration parameter introduced in this version (disabled by default). Doing so causes data nodes to fail whenever they detect a corrupted tuple.

Rows per partition limit removed.  Previously it was possible to store a maximum of 46137488 rows in a single MySQL Cluster partition—that is, per data node. Beginning with MySQL Cluster NDB 7.0.36, this limitation has been lifted, and there is no longer any practical upper limit to this number.

Conflict detection and resolution.  It is now possible to detect and resolve conflicts that arise in multi-master replication scenarios, such as circular replication, when different masters may try to update the same row on the slave with different data. Both “greatest timestamp wins” and “same timestamp wins” scenarios are supported. For more information, see Section 17.6.11, “MySQL Cluster Replication Conflict Resolution”.

Recovery of one master, many slaves replication setups.  Recovery of multi-way replication setups (“one master, many slaves”) is now supported using the --ndb-log-orig server option and changes in the mysql.ndb_binlog_index table. See Section 17.6.4, “MySQL Cluster Replication Schema and Tables”, for more information.

Enhanced selection options for transaction coordinator.  New values and behaviors are introduced for --ndb_optimized_node_selection, enabling greater flexibility when an SQL node chooses a transaction coordinator. For more information, see the description of ndb_optimized_node_selection in Section 17.3.4.3, “MySQL Cluster System Variables”.

Replication heartbeats.  Replication heartbeats facilitate the task of monitoring and detecting failures in master-slave connections in real time. This feature is implemented using a new MASTER_HEARTBEAT_PERIOD = value clause for the CHANGE MASTER TO statement and the addition of two status variables Slave_heartbeat_period and Slave_received_heartbeats. For more information, see Section 13.4.2.1, “CHANGE MASTER TO Syntax”.

NDB thread locks.  It is possible to lock NDB execution threads and maintenance threads (such as file system and other operating system threads) to specific CPUs on multiprocessor data node hosts, and to leverage real-time scheduling.

Improved performance of updates using primary keys or unique keys.  The number of unnecessary reads when performing a primary key or unique key update has been greatly reduced. Since it is seldom necessary to read a record prior to an update, this can yield a considerable improvement in performance. In addition, primary key columns are no longer written to when not needed during update operations.

Batching improvements.  Support of batched DELETE and UPDATE operations has been significantly improved. Batching of UPDATE WHERE... and multiple DELETE operations is also now implemented.

Improved SQL statement performance metrics.  The Ndb_execute_count system status variable measures the number of round trips made by SQL statements to the NDB kernel, providing an improved metric for determining efficiency with which statements are executed. For more information, see MySQL Cluster Status Variables: Ndb_execute_count.

Compressed LCPs and backups.  Compressed local checkpoints and backups can save 50% or more of the disk space used by uncompressed LCPs and backups. These can be enabled using the two new data node configuration parameters CompressedLCP and CompressedBackup, respectively.

OPTIMIZE TABLE support with NDBCLUSTER tables.  OPTIMIZE TABLE is supported for dynamic columns of in-memory NDB tables. In such cases, it is no longer necessary to drop (and possibly to re-create) a table, or to perform a rolling restart, in order to recover memory from deleted rows for general re-use by Cluster. The performance of OPTIMIZE on Cluster tables can be tuned by adjusting the value of the ndb_optimization_delay system variable, which controls the number of milliseconds to wait between processing batches of rows by OPTIMIZE TABLE. In addition, OPTIMIZE TABLE on an NDBCLUSTER table can be interrupted by, for example, killing the SQL thread performing the OPTIMIZE operation.

Batching of transactions.  It is possible to cause statements occurring within the same transaction to be run as a batch by setting the session variable transaction_allow_batching to 1 or ON. To use this feature, autocommit must be set to 0 or OFF. Batch sizes can be controlled using the --ndb-batch-size option for mysqld. For additional information, see Section 17.3.4.2, “MySQL Server Options for MySQL Cluster”, and Section 17.3.4.3, “MySQL Cluster System Variables”.

Attribute promotion with ndb_restore.  It is possible using ndb_restore to restore data reliably from a column of a given type to a column that uses a “larger” type. This is sometimes referred to as attribute promotion. For example, MySQL Cluster backup data that originated in a SMALLINT column can be restored to a MEDIUMINT, INT, or BIGINT column. See Section 17.4.20, “ndb_restore — Restore a MySQL Cluster Backup”, for more information.

Parallel data node recovery.  Recovery of multiple data nodes can now be done in parallel, rather than sequentially. In other words, several data nodes can be restored concurrently, which can often result in much faster recovery times than when they are restored one at a time.

Increased local checkpoint efficiency.  Only 2 local checkpoints are stored, rather than 3, lowering disk space requirements and the size and number of redo log files.

NDBCLUSTER table persistence control.  Persistence of NDB tables can be controlled using the session variables ndb_table_temporary and ndb_table_no_logging. ndb_table_no_logging causes NDB tables not to be checkpointed to disk; ndb_table_temporary does the same, and in addition, no schema files are created. See Section 17.3.4.1, “MySQL Cluster mysqld Option and Variable Reference”.

Epoll support (Linux only).  Epoll is an improved method for handling file descriptors, which is more efficient than scanning to determine whether a file descriptor has data to be read. (The term epoll is specific to Linux and equivalent functionality is known by other names on other platforms such as Solaris and FreeBSD.) Currently, MySQL Cluster supports this functionality on Linux only.

Distribution awareness (SQL nodes).  In MySQL Cluster NDB 6.3, SQL nodes can take advantage of distribution awareness. Here we provide a brief example showing how to design a table to make a given class of queries distrubtion-aware. Suppose an NDBCLUSTER table t1 has the following schema:

CREATE TABLE t1 (
    userid INT NOT NULL,
    serviceid INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
    data VARCHAR(255)
)   ENGINE=NDBCLUSTER;

Suppose further that most of the queries to be used in our application test values of the userid column of this table. The form of such a query looks something like this:

SELECT columns FROM t1
    WHERE userid relation value;

In this query, relation represents some relational operator, such as =, <, >, and so on. Queries using IN and a list of values can also be used:

SELECT columns FROM t1
    WHERE userid IN value_list;

To make use of distribution awareness, we need to make the userid column part of the table's primary key, then explicitly partition the table with this column being used as the partitioning key. (Recall that for a partitioned table having one or more unique keys, all columns of the table's partitioning key must also be part of all of the unique keys—for more information and examples, see Section 18.5.1, “Partitioning Keys, Primary Keys, and Unique Keys”.) In other words, the table schema should be equivalent to the following CREATE TABLE statement:

CREATE TABLE t1 (
    userid INT NOT NULL,
    serviceid INT NOT NULL AUTO_INCREMENT,
    data VARCHAR(255),
    PRIMARY KEY p (userid,serviceid)
)   ENGINE=NDBCLUSTER
    PARTITION BY KEY(userid);

When the table is partitioned in this way, all rows having the same userid value are found on the same node group, and the MySQL Server can immediately select the optimal node to use as the transaction coordinator.

Realtime extensions for multiple CPUs.  When running MySQL Cluster data nodes on hosts with multiple processors, the realtime extensions make it possible to give priority to the data node process and control on which CPU cores it should operate. This can be done using the data node configuration parameters RealtimeScheduler, SchedulerExecutionTimer, and SchedulerSpinTimer. Doing so properly can significantly lower response times and make them much more predictable response. For more information about using these parameters, see Defining Data Nodes: Realtime Performance Parameters

Fully automatic database discovery.  It is no longer a requirement for database autodiscovery that an SQL node already be connected to the cluster at the time that a database is created on another SQL node, or for a CREATE DATABASE or CREATE SCHEMA statement to be issued on the new SQL node after it joins the cluster.

Detection of NDB API client connection errors.  Beginning with MySQL Cluster NDB 6.3.20, the NDB API's Ndb_cluster_connection class adds the get_latest_error() and get_latest_error_msg() methods for catching and diagnosing problems with NDB API client connections.

Restoring specific databases, tables, or columns from a MySQL Cluster backup.  It is now possible to exercise more fine-grained control when restoring a MySQL Cluster from backup using ndb_restore. Beginning with MySQL Cluster NDB 6.3.22, you can choose to restore only specified tables or databases, or exclude specific tables or databases from being restored, using the new ndb_restore options --include-tables, --include-databases, --exclude-tables, and --exclude-databases. Beginning with MySQL Cluster NDB 6.3.26, it is also possible to restore to a table having fewer columns than the original using the --exclude-missing-columns option. For more information about all of these options, see Section 17.4.20, “ndb_restore — Restore a MySQL Cluster Backup”.

Improved Disk Data file system configuration.  As of MySQL Cluster NDB 6.3.22, you can specify default locations for MySQL Cluster Disk Data data files and undo log files using the data node configuration parameters FileSystemPathDD, FileSystemPathDataFiles, and FileSystemPathUndoFiles. This eliminates the need to use symbolic links to place Disk Data files separately from other files in data node file systems to improve Disk Data performance. For more information, see Disk Data file system parameters.

Automatic creation of Disk Data log file groups and tablespaces.  Beginning with MySQL Cluster NDB 6.3.22, using the data node configuration parameters InitialLogFileGroup and InitialTablespace, you can cause the creation of a MySQL Cluster Disk Data log file group, tablespace, or both, when the cluster is first started. When using these parameters, no SQL statements are required to create these Disk Data objects. For more information, see Disk Data object creation parameters.

Configuration parameter data dumps.  Starting with MySQL Cluster NDB 6.3.25, the ndb_config utility supports a --configinfo option that causes it to dump a list of all configuration parameters supported by the cluster, along with brief descriptions, information about the parameters' default and permitted values, and the sections of the config.ini file in which the parameters apply. An additional --xml switch causes ndb_config to use XML rather than plaintext output. Using ndb_config --configinfo or ndb_config --configinfo --xml requires no access to a running MySQL Cluster, any other programs, or any files. For more information and examples, see Section 17.4.7, “ndb_config — Extract MySQL Cluster Configuration Information”.

Per-table reporting of free space on disk.  The INFORMATION_SCHEMA.FILES table shows information about used and free space in MySQL Cluster Disk Data data files, but this information is not applicable to individual tables. In MySQL Cluster NDB 6.3.27 and later, the ndb_desc utility provides two additional columns in its output that show the amount of space allocated on disk for a given NDB table as well the amount of space that remains available for additional storage of disk-based column data for that table. For more information, see Section 17.4.10, “ndb_desc — Describe NDB Tables”.

Improved restart times.  Optimizations in redo log handling and other file system operations introduced in MySQL Cluster NDB 6.3.28 have the potential to reduce considerably the time required for restarts. While actual performance benefits observed in production setups will naturally vary depending on database size, hardware, and other conditions, our own preliminary testing has shown that these improvements can yield startup times that are faster than those typical of previous MySQL Cluster NDB 6.3 releases by a factor of 50 or more.

Increased flexibility in online upgrade procedure.  Previously, when performing an upgrade of a running MySQL cluster, the order in which the types of cluster nodes had to be upgraded was very strict. However, beginning with MySQL Cluster NDB 6.3.29, MySQL Cluster supports online upgrading of API nodes (including MySQL servers running as SQL nodes) before upgrading management nodes, data nodes, or both.

New replication conflict resolution strategy.  Beginning with MySQL Cluster NDB 6.3.31, the function NDB$MAX_DELETE_WIN() is available to implement “greatest timestamp, delete wins” conflict resolution. See NDB$MAX_DELETE_WIN(column_name), for more information.

New CHANGE MASTER TO option for circular replication.  Beginning with MySQL Cluster NDB 6.3.31, the CHANGE MASTER TO statement supports an IGNORE_SERVER_IDS option which takes a comma-separated list of server IDs and causes events originating from the corresponding servers to be ignored. (Log rotation and log deletion events are preserved.)

See Section 13.4.2.1, “CHANGE MASTER TO Syntax”, as well as Section 13.7.5.36, “SHOW SLAVE STATUS Syntax”, for more information.

Heartbeat thread policy and priority.  Beginning with MySQL Cluster NDB 6.3.32, a new configuration parameter HeartbeatThreadPriority makes it possible to set the policy and the priority for the heartbeat thread on management and API nodes.

Improved access to partitioning information.  The ndb_desc utility now provides additional information about the partitioning of data stored in MySQL Cluster. Beginning with MySQL Cluster NDB 6.3.32, the --blob-info option causes this program to include partition information for BLOB tables in its output. Beginning with MySQL Cluster NDB 6.3.33, the --extra-node-info option causes ndb_desc to include information about data distribution (that is, which table fragments are stored on which data nodes). Each of these options also requires the use of the --extra-partition-info option.

Information about partition-to-node mappings can also be obtained using the Table::getFragmentNodes() method, also added in MySQL Cluster NDB 6.3.33.

Replication attribute promotion and demotion.  Beginning with MySQL Cluster NDB 6.3.33, MySQL Cluster Replication supports attribute promotion and demotion when replicating between columns of different but similar types on the master and the slave. For example, it is possible to promote an INT column on the master to a BIGINT column on the slave, and to demote a TEXT column to a VARCHAR column.

The implementation of type demotion distinguishes between lossy and non-lossy type conversions, and their use on the slave can be controlled by setting the slave_type_conversions global server system variable.

For more information, see Attribute promotion and demotion (MySQL Cluster).

Incompatible change in NDB API event reporting.  Beginning with MySQL Cluster NDB 6.3.34, DDL events are no longer reported on Event objects by default. Instead such event reporting must be enabled explicitly using the Event::setReport() method. For more information, see Event::setReport(), and The Event::EventReport Type.

Heartbeat ordering.  Beginning with MySQL Cluster NDB 6.3.35, it is possible to set a specific order for transmission of heartbeats between data nodes, using the HeartbeatOrder data node configuration parameter introduced in this version. This parameter can be useful in situations where multiple data nodes are running on the same host and a temporary disruption in connectivity between hosts would otherwise cause the loss of a node group (and thus failure of the cluster).

Relaxed ndb_restore column comparison rules.  When restoring data, ndb_restore compares the attributes of a column for equality with the definition of the column in the target table. However, not all of these attributes need to be the same for ndb_restore to be meaningful, safe and useful. Beginning with MySQL Cluster NDB 6.3.35, ndb_restore automatically ignores differences in certain column attributes which do not necessarily have to match between the version of the column in a backup and the version of that column in the MySQL Cluster to which the column data is being restored. These attributes include the following:

In such cases, ndb_restore reports any such differences to minimize any chance of user error.

--add-drop-trigger option for mysqldump.  Beginning with MySQL Cluster NDB 6.3.38, this option can be used to force all CREATE TRIGGER statements in mysqldump output to be preceded by a DROP TRIGGER IF EXISTS statement.

Skipping corrupted tables in NDB native backups.  Beginning with MySQL Cluster NDB 6.3.40, you can cause ndb_restore to ignore tables that are corrupted due to missing blob parts tables by using the --skip-broken-objects option. When this option is used, such tables are skipped, and the restoration of any remaining uncorrupted tables in the backup continues.

Restoring from a NDB native backup to a differently-named database.  MySQL Cluster NDB 6.3.41 adds a new --rewrite-database option to ndb_restore, which makes it possible to restore to a database having a different name from that of the database in the backup. The option can be used multiple times, and it is possible to restore from more than one source database in the backup to a single target database (although no protection against table or other object name collision is provided).

See Section 17.4.20, “ndb_restore — Restore a MySQL Cluster Backup”, for more information.

Enhanced backup status reporting.  Backup status reporting has been improved, aided in part by the introduction of a BackupReportFrequency configuration parameter.

Multiple cluster connections per SQL node.  A single MySQL server acting as a MySQL Cluster SQL node can employ multiple connections to the cluster using the --ndb-cluster-connection-pool startup option for mysqld. This option is described in MySQL Cluster-Related Command Options for mysqld: --ndb-cluster-connection-pool option.

New data access interface.  The NdbRecord interface provides a new and simplified data handler for use in NDB API applications.

New reporting commands.  The new management client REPORT BackupStatus and REPORT MemoryUsage commands provide better access to information about the status of MySQL Cluster backups and how much memory is being used by MySQL Cluster for data and index storage. See Section 17.5.2, “Commands in the MySQL Cluster Management Client”, for more information about the REPORT commands. In addition, in-progress status reporting is provided by the ndb_restore utility; see Section 17.4.20, “ndb_restore — Restore a MySQL Cluster Backup”.

Improved memory allocation and configuration.  Memory is now allocated by the NDB kernel to tables on a page-by-page basis, which significantly reduces the memory overhead required for maintaining NDBCLUSTER tables. In addition, the MaxAllocate configuration parameter now makes it possible to set the maximum size of the allocation unit used for table memory.

Choice of fixed-width or variable-width columns.  You can control whether fixed-width or variable-width storage is used for a given column of an NDB table by employing of the COLUMN_FORMAT specifier as part of the column's definition in a CREATE TABLE or ALTER TABLE statement. In addition, the ability to control whether a given column of an NDB table is stored in memory or on disk, using the STORAGE specifier as part of the column's definition in a CREATE TABLE or ALTER TABLE statement. For more information, see Section 13.1.17, “CREATE TABLE Syntax”, and Section 13.1.7, “ALTER TABLE Syntax”.

Controlling management client connections.  The --bind-address cluster management server startup option makes it possible to restrict management client connections to ndb_mgmd to a single host (IP address or host name) and port, which can make MySQL Cluster management operations more secure. For more information about this option, see Section 17.4.4, “ndb_mgmd — The MySQL Cluster Management Server Daemon”.

Micro-GCPs.  Due to a change in the protocol for handling of global checkpoints (GCPs handled in this manner sometimes being referred to as “micro-GCPs”), it is now possible to control how often the GCI number is updated, and how often global checkpoints are written to disk, using the TimeBetweenEpochs and TimeBetweenEpochsTimeout configuration parameters. This improves the reliability and performance of MySQL Cluster Replication.

Core online schema change support.  Support for the online ALTER TABLE operations ADD COLUMN, ADD INDEX, and DROP INDEX is available. When the ONLINE keyword is used, the ALTER TABLE is noncopying, which means that indexes do not have to be re-created, which provides these benefits:

Online CREATE INDEX and DROP INDEX statements are also supported. Online changes can be suppressed using the OFFLINE key word. See Section 13.1.7.2, “ALTER TABLE Online Operations in MySQL Cluster”, Section 13.1.13, “CREATE INDEX Syntax”, and Section 13.1.24, “DROP INDEX Syntax”, for more detailed information.

mysql.ndb_binlog_index improvements.  More information has been added to the mysql.ndb_binlog_index table so that it is possible to determine which originating epochs have been applied inside an epoch. This is particularly useful for 3-way replication. See Section 17.6.4, “MySQL Cluster Replication Schema and Tables”, for more information.

Epoch lag control.  The MaxBufferedEpochs data node configuration parameter provides a means to control the maximum number of unprocessed epochs by which a subscribing node can lag. Subscribers which exceed this number are disconnected and forced to reconnect.

Fully automatic database discovery.  It is no longer a requirement for database autodiscovery that an SQL node already be connected to the cluster at the time that a database is created on another SQL node, or for a CREATE DATABASE or CREATE SCHEMA statement to be issued on the new SQL node after it joins the cluster.

Multiple data node processes per host.  In earlier MySQL Cluster release series, we did not support MySQL Cluster deployments in production where more than one ndbd process was run on a single physical machine. However, beginning with MySQL Cluster NDB 6.2.0, you can use multiple data node processes on a single host.

Improved Disk Data file system configuration.  As of MySQL Cluster NDB 6.2.17, you can specify default locations for MySQL Cluster Disk Data data files and undo log files using the data node configuration parameters FileSystemPathDD, FileSystemPathDataFiles, and FileSystemPathUndoFiles. This eliminates the need to use symbolic links to place Disk Data files separately from other files in data node file systems to improve Disk Data performance. For more information, see Disk Data file system parameters.

Automatic creation of Disk Data log file groups and tablespaces.  Beginning with MySQL Cluster NDB 6.2.17, using the data node configuration parameters InitialLogFileGroup and InitialTablespace, you can cause the creation of a MySQL Cluster Disk Data log file group, tablespace, or both, when the cluster is first started. When using these parameters, no SQL statements are required to create these Disk Data objects.

Improved access to partitioning information.  The ndb_desc utility now provides additional information about the partitioning of data stored in MySQL Cluster. Beginning with MySQL Cluster NDB 6.2.19, the --extra-node-info option causes ndb_desc to include information about data distribution (that is, which table fragments are stored on which data nodes). This option also requires the use of the --extra-partition-info option.

Information about partition-to-node mappings can also be obtained using the Table::getFragmentNodes() method, also added in MySQL Cluster NDB 6.2.19.

New CHANGE MASTER TO option for circular replication.  Beginning with MySQL Cluster NDB 6.2.19, the CHANGE MASTER TO statement supports an IGNORE_SERVER_IDS option which takes a comma-separated list of server IDs and causes events originating from the corresponding servers to be ignored. (Log rotation and log deletion events are preserved.)

See Section 13.4.2.1, “CHANGE MASTER TO Syntax”, as well as Section 13.7.5.36, “SHOW SLAVE STATUS Syntax”, for more information.

Increased number of cluster nodes.  The maximum number of all nodes in a MySQL Cluster has been increased to 255. For more information, see Section 17.1.6.10, “Limitations Relating to Multiple MySQL Cluster Nodes”.

Disabling arbitration.  It is now possible to disable arbitration by setting ArbitrationRank=0 on all cluster management and SQL nodes. For more information, see Defining the Management Server: ArbitrationRank and Defining SQL and Other API Nodes: ArbitrationRank.

Additional DUMP commands.  New management client DUMP commands provide help with tracking transactions, scan operations, and locks. See Section 17.5.2, “Commands in the MySQL Cluster Management Client”, and MySQL Cluster Management Client DUMP Commands, for more information.

Faster Disk Data backups.  Improvements in backups of Disk Data tables can yield a 10 to 15% increase in backup speed of Disk Data tables.

Batched slave updates.  Batching of updates on cluster replication slaves, enabled using the --slave-allow-batching option for mysqld, increases replication efficiency. For more information, see Section 17.6.6, “Starting MySQL Cluster Replication (Single Replication Channel)”.

Integration of MySQL Cluster into MySQL Replication.  MySQL Cluster Replication makes it possible to replicate from one MySQL Cluster to another. Updates on any SQL node (MySQL server) in the cluster acting as the master are replicated to the slave cluster; the state of the slave side remains consistent with the cluster acting as the master. This is sometimes referred to as asynchronous replication between clusters, providing geographic redundancy. It is also possible to replicate from a MySQL Cluster acting as the master to a standalone MySQL server acting as the slave, or from a standalone MySQL master server to a slave cluster; in either of these cases, the standalone MySQL server uses a storage engine other than NDBCLUSTER. Multi-master replication setups such as circular replication are also supported.

See Section 17.6, “MySQL Cluster Replication”.

Support for storage of rows on disk.  Storage of NDBCLUSTER table data on disk is now supported. Indexed columns, including the primary key hash index, must still be stored in RAM; however, all other columns can be stored on disk.

See Section 17.5.12, “MySQL Cluster Disk Data Tables”.

Variable-size columns.  In MySQL 5.0, an NDBCLUSTER table column defined as VARCHAR(255) used 260 bytes of storage independent of what was stored in any particular record. In MySQL 5.1 Cluster tables, only the portion of the column actually taken up by the record is stored. This makes possible a significant reduction in space requirements for such columns as compared to previous release series—by a factor of up to 5 in many cases.

User-defined partitioning.  Users can define partitions based on columns that are part of the primary key. It is possible to partition NDB tables based on KEY and LINEAR KEY schemes. This feature is also available for many other MySQL storage engines, which support additional partitioning types that are not available with NDBCLUSTER tables.

For additional general information about user-defined partitioning in MySQL 5.1, see Chapter 18, Partitioning. Specifics of partitioning types are discussed in Section 18.2, “Partitioning Types”.

The MySQL Server can also determine whether it is possible to “prune away” some of the partitions from the WHERE clause, which can greatly speed up some queries. See Section 18.4, “Partition Pruning”, for information about designing tables and queries to take advantage of partition pruning.

Autodiscovery of table schema changes.  In MySQL 5.0, it was necessary to issue a FLUSH TABLES statement or a “dummy” SELECT for new NDBCLUSTER tables or changes made to schemas of existing NDBCLUSTER tables on one SQL node to be visible on the cluster's other SQL nodes. In MySQL 5.1, this is no longer necessary; new Cluster tables and changes in the definitions of existing NDBCLUSTER tables made on one SQL node are immediately visible to all SQL nodes connected to the cluster.

Distribution awareness (NDB API).  Distribution awareness is a mechanism by which the best data node is automatically selected to be queried for information. (Conceptually, it is similar in some ways to partition pruning (see Section 18.4, “Partition Pruning”). To take advantage of distribution awareness, you should do the following:

Following these steps causes records with the same value for the partitioning column to be stored on the same partition (that is, in the same node group). When reading data, transactions are begun on the data node actually having the desired rows instead of this node being determined by the usual round-robin mechanism.

Using distribution awareness can yield performance increase of as great as 45% when using four data nodes, and possibly more when using a greater number of data nodes.

Online adding and dropping of indexes.  For ALTER TABLE statements, ADD INDEX and DROP INDEX operations for dynamic (variable-width) columns on NDB tables are now performed as online operations (no table copying). This is also true for CREATE INDEX and DROP INDEX. As a result, these operations are now performed much more quickly than previously.

For more information, see Section 13.1.7.2, “ALTER TABLE Online Operations in MySQL Cluster”, Section 13.1.13, “CREATE INDEX Syntax”, and Section 13.1.24, “DROP INDEX Syntax”.

Temporary tables.  Temporary tables are not supported. Trying either to create a temporary table that uses the NDB storage engine or to alter an existing temporary table to use NDB fails with the error Table storage engine 'ndbcluster' does not support the create option 'TEMPORARY'.

Indexes and keys in NDB tables.  Keys and indexes on MySQL Cluster tables are subject to the following limitations:

MySQL Cluster and geometry data types.  Geometry data types (WKT and WKB) are supported in NDB tables in MySQL 5.1 (including MySQL Cluster NDB 6.X and 7.X through 7.1). However, spatial indexes are not supported.

Character sets and binary log files.  Currently, the ndb_apply_status and ndb_binlog_index tables are created using the latin1 (ASCII) character set. Because names of binary logs are recorded in this table, binary log files named using non-Latin characters are not referenced correctly in these tables. This is a known issue, which we are working to fix. (Bug #50226)

To work around this problem, use only Latin-1 characters when naming binary log files or setting any the --basedir, --log-bin, or --log-bin-index options.

Creating NDBCLUSTER tables with user-defined partitioning.  Support for user-defined partitioning for MySQL Cluster is restricted to [LINEAR] KEY partitioning. Beginning with MySQL 5.1.12, using any other partitioning type with ENGINE=NDB or ENGINE=NDBCLUSTER in a CREATE TABLE statement results in an error.

Default partitioning scheme.  As of MySQL 5.1.6, all MySQL Cluster tables are by default partitioned by KEY using the table's primary key as the partitioning key. If no primary key is explicitly set for the table, the “hidden” primary key automatically created by the NDBCLUSTER storage engine is used instead. For additional discussion of these and related issues, see Section 18.2.4, “KEY Partitioning”.

Beginning with MySQL Cluster NDB 6.2.18, MySQL Cluster NDB 6.3.25, and MySQL Cluster NDB 7.0.6, CREATE TABLE and ALTER TABLE statements that would cause a user-partitioned NDBCLUSTER table not to meet either or both of the following two requirements are not permitted, and fail with an error (Bug #40709):

Exception.  If a user-partitioned NDBCLUSTER table is created using an empty column-list (that is, using PARTITION BY [LINEAR] KEY()), then no explicit primary key is required.

Maximum number of partitions for NDBCLUSTER tables.  The maximum number of partitions that can defined for a NDBCLUSTER table when employing user-defined partitioning is 8 per node group. (See Section 17.1.2, “MySQL Cluster Nodes, Node Groups, Replicas, and Partitions”, for more information about MySQL Cluster node groups.

DROP PARTITION not supported.  It is not possible to drop partitions from NDB tables using ALTER TABLE ... DROP PARTITION. The other partitioning extensions to ALTER TABLE—ADD PARTITION, REORGANIZE PARTITION, and COALESCE PARTITION—are supported for Cluster tables, but use copying and so are not optimized. See Section 18.3.1, “Management of RANGE and LIST Partitions” and Section 13.1.7, “ALTER TABLE Syntax”.

Row-based replication.  When using row-based replication with MySQL Cluster, binary logging cannot be disabled. That is, the NDB storage engine ignores the value of sql_log_bin. (Bug #16680)

A DELETE statement on an NDB table makes the memory formerly used by the deleted rows available for re-use by inserts on the same table only. However, this memory can be made available for general re-use by performing a rolling restart of the cluster. See Section 17.5.5, “Performing a Rolling Restart of a MySQL Cluster”.

Beginning with MySQL Cluster NDB 6.3.7, this limitation can be overcome using OPTIMIZE TABLE. See Section 17.1.6.11, “Previous MySQL Cluster Issues Resolved in MySQL 5.1, MySQL Cluster NDB 6.x, and MySQL Cluster NDB 7.x”, for more information.

A DROP TABLE or TRUNCATE TABLE operation on an NDB table frees the memory that was used by this table for re-use by any NDB table, either by the same table or by another NDB table.

Limits imposed by the cluster's configuration.  A number of hard limits exist which are configurable, but available main memory in the cluster sets limits. See the complete list of configuration parameters in Section 17.3.2, “MySQL Cluster Configuration Files”. Most configuration parameters can be upgraded online. These hard limits include:

Node and data object maximums.  The following limits apply to numbers of cluster nodes and metadata objects:

See Section 17.1.6.11, “Previous MySQL Cluster Issues Resolved in MySQL 5.1, MySQL Cluster NDB 6.x, and MySQL Cluster NDB 7.x”, for more information.

Transaction isolation level.  The NDBCLUSTER storage engine supports only the READ COMMITTED transaction isolation level. (InnoDB, for example, supports READ COMMITTED, READ UNCOMMITTED, REPEATABLE READ, and SERIALIZABLE.) See Section 17.5.3.4, “MySQL Cluster Backup Troubleshooting”, for information on how this can affect backing up and restoring Cluster databases.)

Transactions and BLOB or TEXT columns.  NDBCLUSTER stores only part of a column value that uses any of MySQL's BLOB or TEXT data types in the table visible to MySQL; the remainder of the BLOB or TEXT is stored in a separate internal table that is not accessible to MySQL. This gives rise to two related issues of which you should be aware whenever executing SELECT statements on tables that contain columns of these types:

Rollbacks.  There are no partial transactions, and no partial rollbacks of transactions. A duplicate key or similar error causes the entire transaction to be rolled back.

This behavior differs from that of other transactional storage engines such as InnoDB that may roll back individual statements.

Transactions and memory usage.  As noted elsewhere in this chapter, MySQL Cluster does not handle large transactions well; it is better to perform a number of small transactions with a few operations each than to attempt a single large transaction containing a great many operations. Among other considerations, large transactions require very large amounts of memory. Because of this, the transactional behavior of a number of MySQL statements is effected as described in the following list:

Transactions and the COUNT() function.  When using MySQL Cluster Replication, it is not possible to guarantee the transactional consistency of the COUNT() function on the slave. In other words, when performing on the master a series of statements (INSERT, DELETE, or both) that changes the number of rows in a table within a single transaction, executing SELECT COUNT(*) FROM table queries on the slave may yield intermediate results. This is due to the fact that SELECT COUNT(...) may perform dirty reads, and is not a bug in the NDB storage engine. (See Bug #31321 for more information.)

Temporary errors.  When first starting a node, it is possible that you may see Error 1204 Temporary failure, distribution changed and similar temporary errors.

Errors due to node failure.  The stopping or failure of any data node can result in a number of different node failure errors. (However, there should be no aborted transactions when performing a planned shutdown of the cluster.)

Table names containing special characters.  NDB tables whose names contain characters other than letters, numbers, dashes, and underscores and which are created on one SQL node were not always discovered correctly by other SQL nodes. (Bug #31470)

Number of database objects.  The maximum number of all NDB database objects in a single MySQL Cluster—including databases, tables, and indexes—is limited to 20320.

Attributes per table.  Prior to MySQL Cluster NDB 7.0.15 and MySQL Cluster NDB 7.1.4, the maximum number of attributes (that is, columns and indexes) per table is limited to 128.

Beginning with MySQL Cluster NDB 7.0.15 and MySQL Cluster NDB 7.1.4, this limit is increased to 512.

Attributes per key.  The maximum number of attributes per key is 32.

Row size.  Prior to MySQL Cluster NDB 7.0, the maximum permitted size of any one row was 8052 bytes; in MySQL Cluster NDB 7.0 and later, this is 14000 bytes. Each BLOB or TEXT column contributes 256 + 8 = 264 bytes to this total.

Foreign key constraints.  Prior to MySQL Cluster NDB 7.3, the foreign key construct is ignored, just as it is by MyISAM tables. Foreign keys are supported in MySQL Cluster NDB 7.3 and later.

Index prefixes.  Prefixes on indexes are not supported for NDBCLUSTER tables. If a prefix is used as part of an index specification in a statement such as CREATE TABLE, ALTER TABLE, or CREATE INDEX, the prefix is ignored.

OPTIMIZE operations.  OPTIMIZE operations are not supported.

Beginning with MySQL Cluster NDB 6.3.7, this limitation has been lifted. See Section 17.1.6.11, “Previous MySQL Cluster Issues Resolved in MySQL 5.1, MySQL Cluster NDB 6.x, and MySQL Cluster NDB 7.x”, for more information.

LOAD TABLE ... FROM MASTER.  LOAD TABLE FROM MASTER is not supported.

Savepoints and rollbacks.  Savepoints and rollbacks to savepoints are ignored as in MyISAM.

Durability of commits.  There are no durable commits on disk. Commits are replicated, but there is no guarantee that logs are flushed to disk on commit.

Replication.  Statement-based replication is not supported. Use --binlog-format=ROW (or --binlog-format=MIXED) when setting up cluster replication. See Section 17.6, “MySQL Cluster Replication”, for more information.

Range scans.  There are query performance issues due to sequential access to the NDB storage engine; it is also relatively more expensive to do many range scans than it is with either MyISAM or InnoDB.

Reliability of Records in range.  The Records in range statistic is available but is not completely tested or officially supported. This may result in nonoptimal query plans in some cases. If necessary, you can employ USE INDEX or FORCE INDEX to alter the execution plan. See Section 13.2.8.3, “Index Hint Syntax”, for more information on how to do this.

Unique hash indexes.  Unique hash indexes created with USING HASH cannot be used for accessing a table if NULL is given as part of the key.

Machine architecture.  All machines used in the cluster must have the same architecture. That is, all machines hosting nodes must be either big-endian or little-endian, and you cannot use a mixture of both. For example, you cannot have a management node running on a PowerPC which directs a data node that is running on an x86 machine. This restriction does not apply to machines simply running mysql or other clients that may be accessing the cluster's SQL nodes.

Binary logging.  MySQL Cluster has the following limitations or restrictions with regard to binary logging:

Maximum number of tablespaces: 2³² (4294967296)

Maximum number of data files per tablespace: 2¹⁶ (65536)

Maximum data file size: The theoretical limit is 64G; however, in MySQL 5.1 (including MySQL Cluster NDB 6.X and 7.X through 7.1), the practical upper limit is 32G. This is equivalent to 32768 extents of 1M each.

Since a MySQL Cluster Disk Data table can use at most 1 tablespace, this means that the theoretical upper limit to the amount of data (in bytes) that can be stored on disk by a single NDB table is 32G * 65536 = 2251799813685248, or approximately 2 petabytes.

The theoretical maximum number of extents per tablespace data file is 2¹⁶ (65536); however, for practical purposes, the recommended maximum number of extents per data file is 2¹⁵ (32768).

The minimum and maximum possible sizes of extents for tablespace data files are 32K and 2G, respectively. See Section 13.1.18, “CREATE TABLESPACE Syntax”, for more information.

No distributed table locks.  A LOCK TABLES works only for the SQL node on which the lock is issued; no other SQL node in the cluster “sees” this lock. This is also true for a lock issued by any statement that locks tables as part of its operations. (See next item for an example.)

ALTER TABLE operations.  ALTER TABLE is not fully locking when running multiple MySQL servers (SQL nodes). (As discussed in the previous item, MySQL Cluster does not support distributed table locks.)

You must give nodes explicit IDs in connect strings because automatic allocation of node IDs does not work across multiple management servers.

In MySQL Cluster NDB 7.0.7 and earlier, you must take extreme care to have the same configurations for all management servers; no special checks for consistency are performed.

Beginning with MySQL Cluster NDB 7.0.8, a management server when first starting checks for any other management server in the same MySQL Cluster, and upon successful connection to the other management server uses its configuration data. This means that the management server --reload and --initial startup options are ignored unless the management server is the only one running. It also means that, when performing a rolling restart of a MySQL Cluster with multiple management nodes, the management server reads its own configuration file if (and only if) it is the only management server running in this MySQL Cluster. See Section 17.5.5, “Performing a Rolling Restart of a MySQL Cluster”, for more information.

Variable-length column support.  The NDBCLUSTER storage engine now supports variable-length column types for in-memory tables.

Previously, for example, any Cluster table having one or more VARCHAR fields which contained only relatively small values, much more memory and disk space were required when using the NDBCLUSTER storage engine than would have been the case for the same table and data using the MyISAM engine. In other words, in the case of a VARCHAR column, such a column required the same amount of storage as a CHAR column of the same size. In MySQL 5.1, this is no longer the case for in-memory tables, where storage requirements for variable-length column types such as VARCHAR and BINARY are comparable to those for these column types when used in MyISAM tables (see Section 11.7, “Data Type Storage Requirements”).

Replication with MySQL Cluster.  It is now possible to use MySQL replication with Cluster databases. For details, see Section 17.6, “MySQL Cluster Replication”.

Circular Replication.  Circular replication is also supported with MySQL Cluster, beginning with MySQL 5.1.18. See Section 17.6.10, “MySQL Cluster Replication: Multi-Master and Circular Replication”.

auto_increment_increment and auto_increment_offset.  The auto_increment_increment and auto_increment_offset server system variables are supported for Cluster replication beginning with MySQL 5.1.20, MySQL Cluster NDB 6.2.5, and MySQL Cluster 6.3.2.

Database autodiscovery and online schema changes.  Autodiscovery of databases is now supported for multiple MySQL servers accessing the same MySQL Cluster. Formerly, autodiscovery in MySQL Cluster 5.1 and MySQL Cluster NDB 6.x releases required that a given mysqld was already running and connected to the cluster at the time that the database was created on a different mysqld—in other words, when a mysqld process connected to the cluster after a database named db_name was created, it was necessary to issue a CREATE DATABASE db_name or CREATE SCHEMA db_name statement on the “new” MySQL server when it first accessed that MySQL Cluster. Beginning with MySQL Cluster NDB 6.2.16 and MySQL Cluster NDB 6.3.18, such a CREATE statement is no longer required. (Bug #39612)

This also means that online schema changes in NDB tables are now possible. That is, the result of operations such as ALTER TABLE and CREATE INDEX performed on one SQL node in the cluster are now visible to the cluster's other SQL nodes without any additional action being taken.

Backup and restore between architectures.  Beginning with MySQL 5.1.10, it is possible to perform a Cluster backup and restore between different architectures. Previously—for example—you could not back up a cluster running on a big-endian platform and then restore from that backup to a cluster running on a little-endian system. (Bug #19255)

Character set directory.  Beginning with MySQL 5.1.10, it is possible to install MySQL with Cluster support to a nondefault location and change the search path for font description files using either the --basedir or --character-sets-dir options. (Previously, ndbd in MySQL 5.1 searched only the default path—typically /usr/local/mysql/share/mysql/charsets—for character sets.)

Multiple management servers.  In MySQL 5.1 (including all MySQL Cluster NDB 6.x and later versions), it is no longer necessary, when running multiple management servers, to restart all the cluster's data nodes to enable the management nodes to see one another.

Also, when using multiple management servers and starting concurrently several API nodes (possibly including one or more SQL nodes) whose connection strings listed the management servers in different order, it was possible for 2 API nodes to be assigned the same node ID. This issue is resolved in MySQL Cluster NDB 6.2.17, 6.3.23, and 6.4.3. (Bug #42973)

Multiple data node processes per host.  Beginning with MySQL Cluster NDB 6.2.0, you can use multiple data node processes on a single host. (In MySQL Cluster NDB 6.1, MySQL 5.1, and earlier release series, we did not support production MySQL Cluster deployments in which more than one ndbd process was run on a single physical machine.)

In addition, MySQL Cluster NDB 7.0 introduces support for multi-threaded data nodes (ndbmtd). See Section 17.1.4.3, “MySQL Cluster Development in MySQL Cluster NDB 7.0”, and Section 17.4.3, “ndbmtd — The MySQL Cluster Data Node Daemon (Multi-Threaded)”, for more information.

Identifiers.  Formerly (in MySQL 5.0 and earlier), database names, table names and attribute names could not be as long for NDB tables as tables using other storage engines, because attribute names were truncated internally. In MySQL 5.1 and later, names of MySQL Cluster databases, tables, and table columns follow the same rules regarding length as they do for any other storage engine.

Length of CREATE TABLE statements.  CREATE TABLE statements may be no more than 4096 characters in length. This limitation affects MySQL 5.1.6, 5.1.7, and 5.1.8 only. (See Bug #17813)

IGNORE and REPLACE functionality.  In MySQL 5.1.7 and earlier, INSERT IGNORE, UPDATE IGNORE, and REPLACE were supported only for primary keys, but not for unique keys. It was possible to work around this issue by removing the constraint, then dropping the unique index, performing any inserts, and then adding the unique index again.

This limitation was removed for INSERT IGNORE and REPLACE in MySQL 5.1.8. (See Bug #17431.)

AUTO_INCREMENT columns.  In MySQL 5.1.10 and earlier versions, the maximum number of tables having AUTO_INCREMENT columns—including those belonging to hidden primary keys—was 2048.

This limitation was lifted in MySQL 5.1.11.

Maximum number of cluster nodes.  Prior to MySQL Cluster NDB 6.1.1, the total maximum number of nodes in a MySQL Cluster was 63, including all SQL nodes (MySQL Servers), API nodes (applications accessing the cluster other than MySQL servers), data nodes, and management servers.

Starting with MySQL Cluster NDB 6.1.1, the total maximum number of nodes in a MySQL Cluster is 255, including all SQL nodes (MySQL Servers), API nodes (applications accessing the cluster other than MySQL servers), data nodes, and management servers. The total number of data nodes and management nodes beginning with this version is 63, of which up to 48 can be data nodes.

Recovery of memory from deleted rows.  Beginning with MySQL Cluster NDB 6.3.7, memory can be reclaimed from an NDB table for reuse with any NDB table by employing OPTIMIZE TABLE, subject to the following limitations:

You can regulate the effects of OPTIMIZE on performance by adjusting the value of the global system variable ndb_optimization_delay, which sets the number of milliseconds to wait between batches of rows being processed by OPTIMIZE. The default value is 10 milliseconds. It is possible to set a lower value (to a minimum of 0), but not recommended. The maximum is 100000 milliseconds (that is, 100 seconds).

Implicit Rollbacks.  Prior to MySQL Cluster NDB 6.2.17 and MySQL Cluster NDB 6.3.19, MySQL Cluster did not automatically roll back a transaction that was aborted by a duplicate key or similar error, and subsequent statements raised ERROR 1296 (HY000): Got error 4350 'Transaction already aborted' from NDBCLUSTER. In such cases, it was necessary to issue an explicit ROLLBACK statement first, and then to retry the entire transaction.

Beginning with MySQL Cluster NDB 6.2.17 and MySQL Cluster NDB 6.3.19, this limitation has been removed; now, an error which causes a transaction to be aborted generates an implicit rollback of the entire transaction. This is logged with the warning Storage engine NDB does not support rollback for this statement. Transaction rolled back and must be restarted. A statement subsequent to this starts a new transaction. (Bug #32656)

Number of tables.  Previously, the maximum number of NDBCLUSTER tables in a single MySQL Cluster was 1792, but this is no longer the case in MySQL 5.1 and later MySQL Cluster releases. However, the number of tables is still included in the total maximum number of NDBCLUSTER database objects (20320). (See Section 17.1.6.5, “Limits Associated with Database Objects in MySQL Cluster”.)

DDL operations.  Beginning with MySQL Cluster NDB 6.4.0, DDL operations (such as CREATE TABLE or ALTER TABLE) are protected from data node failures. Previously, if a data node failed while trying to perform one of these, the data dictionary became locked and no further DDL statements could be executed without restarting the cluster (Bug #36718).

Adding and dropping of data nodes.  In MySQL Cluster NDB 6.3 and previous versions of MySQL Cluster, the online adding or dropping of data nodes was not possible; such operations required a complete shutdown and restart of the entire cluster. In MySQL Cluster NDB 7.0 (beginning with MySQL Cluster NDB 6.4.0) and later MySQL Cluster release series, it is possible to add new data nodes to a running MySQL Cluster by performing a rolling restart, so that the cluster and the data stored in it remain available to applications.

When planning to increase the number of data nodes in the cluster online in MySQL Cluster NDB 7.0 or MySQL Cluster NDB 7.1, you should be aware of and take into account the following issues:

For more information, see Section 17.5.13, “Adding MySQL Cluster Data Nodes Online”.

Native support for default column values.  Starting with MySQL Cluster NDB 7.1.0, default values for table columns are stored by NDBCLUSTER, rather than by the MySQL server as was previously the case. Because less data must be sent from an SQL node to the data nodes, inserts on tables having column value defaults can be performed more efficiently than before.

Tables created using previous MySQL Cluster releases can still be used in MySQL Cluster 7.1.0 and later, although they do not support native default values and continue to use defaults supplied by the MySQL server until they are upgraded. This can be done by means of an offline ALTER TABLE statement.

InnoDB plugin support.  Previously, InnoDB support in MySQL Cluster was limited to the version built in to the MySQl Server. Beginning with MySQL Cluster NDB 7.1.9, MySQL Cluster also provides support for the InnoDB Plugin. See Section 17.2, “MySQL Cluster Installation and Upgrades”, for information about enabling InnoDB storage engine and plugin support with MySQL Cluster.

Distribution of MySQL users and privileges.  Previously, MySQL users and privileges created on one SQL node were unique to that SQL node, due to the fact that the MySQL grant tables were restricted to using the MyISAM storage engine. Beginning with MySQL Cluster NDB 7.2.0, it is possible, following installation of the MySQL Cluster software and setup of the desired users and privileges on one SQL node, to convert the grant tables to use NDB and thus to distribute the users and privileges across all SQL nodes connected to the cluster. You can do this by loading and making use of a set of stored procedures defined in an SQL script supplied with the MySQL Cluster distribution. For more information, see Section 17.5.14, “Distributed MySQL Privileges for MySQL Cluster”.

Number of rows per partition.  Previously, a single MySQL Cluster partition could hold a maximum of 46137488 rows. This limitation was removed in MySQL Cluster NDB 7.0.36 and MySQL Cluster NDB 7.1.25. (Bug #13844405, Bug #14000373)

If you are still using a previous MySQL Cluster release, you can work around this limitation by taking advantage of the fact that the number of partitions is the same as the number of data nodes in the cluster (see Section 17.1.2, “MySQL Cluster Nodes, Node Groups, Replicas, and Partitions”). This means that, by increasing the number of data nodes, you can increase the available space for storing data.

In MySQL Cluster NDB 7.0 and later, you can increase the number of data nodes in the cluster while the cluster remains in operation. See Section 17.5.13, “Adding MySQL Cluster Data Nodes Online”, for more information.

It is also possible to increase the number of partitions for NDB tables by using explicit KEY or LINEAR KEY partitioning (see Section 18.2.4, “KEY Partitioning”).

The Server RPM (for example, MySQL-Cluster-gpl-server-6.3.55-0.sles10.i586.rpm, MySQL-Cluster-gpl-server-7.0.42-0.sles10.i586.rpm, or MySQL-Cluster-gpl-server-7.1.34-0.sles10.i586.rpm), which supplies the core files needed to run a MySQL Server with NDBCLUSTER storage engine support (that is, as a MySQL Cluster SQL node).

If you do not have your own client application capable of administering a MySQL server, you should also obtain and install the Client RPM (for example, MySQL-Cluster-gpl-client-6.3.55-0.sles10.i586.rpm, MySQL-Cluster-gpl-client-7.0.42-0.sles10.i586.rpm, or MySQL-Cluster-gpl-client-7.1.34-0.sles10.i586.rpm).

The Cluster storage engine RPM (for example, MySQL-Cluster-gpl-storage-6.3.55-0.sles10.i586.rpm, MySQL-Cluster-gpl-storage-7.0.42-0.sles10.i586.rpm, or MySQL-Cluster-gpl-storage-7.1.34-0.sles10.i586.rpm), which supplies the MySQL Cluster data node binary (ndbd).

The Cluster storage engine management RPM (for example, MySQL-Cluster-gpl-management-6.3.55-0.sles10.i586.rpm, MySQL-Cluster-gpl-management-7.0.42-0.sles10.i586.rpm, or MySQL-Cluster-gpl-management-7.1.34-0.sles10.i586.rpm) which provides the MySQL Cluster management server binary (ndb_mgmd).

To build MySQL Cluster, you must use the MySQL Cluster sources, which you can obtain from http://dev.mysql.com/downloads/cluster/.

Attempting to build MySQL Cluster from the source code for the standard MySQL Server is likely not to be successful, and is not supported by Oracle.

You must configure the build using the WITH_NDBCLUSTER_STORAGE_ENGINE option in addition to any other build options you wish to use before creating the Visual Studio project files. Once you have run configure.js with the desired options, you can create the project files and build from them in the same manner as you do when compiling the standard MySQL Server. For more information, see Section 2.11.7, “Installing MySQL from Source on Windows”.

Prior to MySQL Cluster NDB 7.0.7, DML statements failed if executed while performing an online upgrade from a MySQL Cluster NDB 6.3 release. (Bug #45917)

Following an upgrade from any MySQL Cluster NDB 6.3.x release to MySQL Cluster NDB 7.0.6, DDL and backup operations failed. This issue was resolved in MySQL Cluster NDB 7.0.7. (Bug #46494, Bug #46563)

In some cases, there could be problems with online upgrades from MySQL Cluster NDB 6.3 releases to MySQL Cluster NDB 7.0 releases due to a previous change in the signalling format used between nodes. This issue was corrected in MySQL Cluster NDB 7.0.9.

Once an NDB table had an ALTER ONLINE TABLE operation performed on it using a MySQL Cluster NDB 6.3.x release, it could not be upgraded online to MySQL Cluster NDB 7.0. This issue was resolved in MySQL Cluster NDB 7.0.8. (See Bug #47542.)

Following an upgrade from MySQL Cluster NDB 6.3 to MySQL Cluster NDB 7.0, if there were any tables having unique indexes prior to the upgrade, attempts to create unique indexes failed. This could also occur when performing offline ALTER TABLE operations on tables having indexes that were not dropped as a result of the ALTER TABLE. This issue was due to a change in the way that NDB tracked unique indexes internally, and was resolved in MySQL Cluster NDB 7.0.9. (Bug #48416)

For upgrades to MySQL Cluster NDB 7.0 releases prior to version 7.0.9, a workaround is available: Following the upgrade, execute a second rolling restart of the cluster before performing any ALTER TABLE operations involving indexes.

Due to an issue discovered after the release of MySQL Cluster NDB 7.0.10 (Bug #50433), it is not possible to perform an online upgrade from MySQL Cluster NDB 7.0.9b and earlier MySQL Cluster NDB 7.0 releases to MySQL Cluster NDB 7.0.10. Instead, you should upgrade your MySQL Cluster NDB 7.0 installation directly to MySQL Cluster NDB 7.0.11 or later.

This issue did not appear to affect MySQL Cluster NDB 6.3, and it should be possible to upgrade online from MySQL Cluster NDB 6.3 to MySQL Cluster NDB 7.0.10 without any problems other than those noted previously.

It was not possible to perform an online upgrade from a MySQL Cluster NDB 6.3 or 7.0 release to MySQL Cluster NDB 7.1.0 or 7.1.1. This issue was fixed in MySQL Cluster NDB 7.1.2 (see Bug #51429).

Following an upgrade to MySQL Cluster NDB 7.0.15 or later (MySQL Cluster NDB 7.0), or to MySQL Cluster NDB 7.1.4 or later (MySQL Cluster NDB 7.1), a table created in a previous version of MySQL Cluster does not automatically support NDB-native default values. Such a table continues to use default values supplied by the MySQL server until it is upgraded by performing an offline ALTER TABLE on it.

When upgrading to a MySQL Cluster release that supports native default values from a MySQL Cluster release that does not, you should not attempt to create any new tables until all data nodes are using the new ndbd or ndbmtd binaries. This is because the older binaries do not provide support for native default values.

NDB API changes in MySQL Cluster NDB 7.1.16 and MySQL Cluster NDB 7.0.27 are not backward compatible. Due to these changes the version of the included NDB client library was increased from 4.0.0 to 5.0.0; NDB API applications must be recompiled as part of any upgrade to these or later MySQL Cluster releases.

In addition, NDB API programs should be updated as part of such an upgrade, in order to take into account changes in the default behavior and values for the Ndb_cluster_connection::connect() method.

Due to issues discovered shortly after release, MySQL Cluster NDB 7.0.20 was withdrawn and replaced with MySQL Cluster NDB 7.0.20a. Users of MySQL Cluster NDB 7.0.19 and previous MySQL Cluster NDB 7.0 releases should upgrade to MySQL Cluster NDB 7.0.20a, or to a later MySQL Cluster NDB 7.0 release. See http://dev.mysql.com/doc/relnotes/mysql-cluster/7.1/en/mysql-cluster-news-5-1-51-ndb-7-0-20a.html, for more information.

Due to issues discovered following release, MySQL Cluster NDB 7.1.15 was withdrawn and replaced with MySQL Cluster NDB 7.1.15a. Users of MySQL Cluster NDB 7.1.14 and previous MySQL Cluster NDB 7.1 releases should upgrade to MySQL Cluster NDB 7.1.15a, or to a later MySQL Cluster NDB 7.1 release. See http://dev.mysql.com/doc/relnotes/mysql-cluster/7.1/en/mysql-cluster-news-5-1-56-ndb-7-1-15a.html, for more information.

Due to issues discovered shortly after release, MySQL Cluster NDB 7.1.9 was withdrawn and replaced with MySQL Cluster NDB 7.1.9a. Users of MySQL Cluster NDB 7.1.8 and previous MySQL Cluster NDB 7.1 releases should upgrade to MySQL Cluster NDB 7.1.14 (the last release directly upgrade-compatible with MySQL Cluster 7.1.18 and earlier) before upgrading to MySQL Cluster 7.1.15a or later. See http://dev.mysql.com/doc/relnotes/mysql-cluster/7.1/en/mysql-cluster-news-5-1-51-ndb-7-1-9a.html, for more information.

When performing an online upgrade or downgrade between MySQL Cluster NDB 7.1.8 or earlier and a later release up to and including MySQL Cluster NDB 7.1.14, you must upgrade or downgrade the data nodes before upgrading or downgrading any SQL nodes; otherwise mysql_upgrade fails on the SQL nodes due to differences between ndbinfo tables used in the “old” and “new” versions of the MySQL Cluster software. You should also upgrade or downgrade the data nodes prior to the SQL nodes when performing an online upgrade or downgrade between MySQL Cluster NDB 7.1 releases where either of the versions involved is MySQL Cluster NDB 7.1.14 or earlier, and where one or more ndbinfo tables has more, fewer, or differing columns between the two versions.

This issue is resolved in MySQL Cluster NDB 7.1.15. (Bug #11885602)

It was not possible to downgrade online to MySQL Cluster NDB 7.1 from some later versions of MySQL Cluster due to a change in the size used for table hash maps by NDB. (Bug #14645319) This issue was resolved in MySQL Cluster NDB 7.1.26, where the size is made configurable using the DefaultHashMapSize parameter. (Bug #14800539) See the description of this parameter for more information.

It is not possible to upgrade from MySQL Cluster NDB 6.1.2 (or an older 6.1 release) directly to 6.1.4 or a newer NDB 6.1 release, or to downgrade from 6.1.4 (or a newer 6.1 release) directly to 6.1.2 or an older NDB 6.1 release; in either case, you must upgrade or downgrade to MySQL Cluster NDB 6.1.3 first.

It is not possible to perform an online downgrade from MySQL Cluster NDB 6.1.8 (or a newer 6.1 release) to MySQL Cluster NDB 6.1.7 (or an older 6.1 release).

MySQL Cluster NDB 6.1.6 and 6.1.18 were not released.

It is not possible to perform an online upgrade or downgrade between MySQL Cluster NDB 6.2 and any previous release series (including mainline MySQL 5.1 and MySQL Cluster NDB 6.1); it is necessary to perform a dump and reload. However, it should be possible to perform online upgrades or downgrades between any MySQL Cluster NDB 6.2 release and any MySQL Cluster NDB 6.3 release up to and including 6.3.7.

MySQL Cluster NDB 6.1.8 or a later MySQL Cluster NDB 6.1 release

MySQL Cluster 6.2.1 or a later MySQL Cluster NDB 6.2 release

Any MySQL Cluster NDB 6.3 release (or later MySQL Cluster release series)

It was not possible to perform an online upgrade between any MySQL Cluster NDB 6.2 release and MySQL Cluster NDB 6.3.8 and later MySQL Cluster 6.3 releases. This issue was fixed in MySQL Cluster NDB 6.3.21. (Bug #41435)

Online downgrades between MySQL Cluster NDB 6.2.5 and earlier releases are not supported.

Online downgrades between MySQL Cluster NDB 6.3.8 and earlier releases are not supported.

MySQL 5.1.3 was the first public release in this series.

Direct upgrades or downgrades between MySQL Cluster 5.0 and 5.1 are not supported; you must dump all NDBCLUSTER tables using mysqldump, install the new version of the software, and then reload the tables from the dump.

You cannot downgrade a MySQL Cluster based on MySQL 5.1.6 or later and using Disk Data tables to MySQL 5.1.5 or earlier unless you convert all such tables to in-memory NDB tables first.

MySQL 5.1.8, MySQL 5.1.10, and MySQL 5.1.13 were not released.

Online cluster upgrades and downgrades between MySQL 5.1.11 (or an earlier version) and 5.1.12 (or a later version) are not possible due to major changes in the cluster file system. In such cases, you must perform a backup or dump, upgrade (or downgrade) the software, start each data node with --initial, and then restore from the backup or dump. You can use native NDB backup and restore, or mysqldump and LOAD DATA INFILE for this purpose.

Online downgrades from MySQL 5.1.14 or later to versions previous to 5.1.14 are not supported due to incompatible changes in the cluster system tables.

[computer]: Defines cluster hosts. This is not required to configure a viable MySQL Cluster, but be may used as a convenience when setting up a large cluster. See Section 17.3.2.4, “Defining Computers in a MySQL Cluster”, for more information.

[ndbd]: Defines a cluster data node (ndbd process). See Section 17.3.2.6, “Defining MySQL Cluster Data Nodes”, for details.

[mysqld]: Defines the cluster's MySQL server nodes (also called SQL or API nodes). For a discussion of SQL node configuration, see Section 17.3.2.7, “Defining SQL and Other API Nodes in a MySQL Cluster”.

[mgm] or [ndb_mgmd]: Defines a cluster management server (MGM) node. For information concerning the configuration of management nodes, see Section 17.3.2.5, “Defining a MySQL Cluster Management Server”.

[tcp]: Defines a TCP/IP connection between cluster nodes, with TCP/IP being the default connection protocol. Normally, [tcp] or [tcp default] sections are not required to set up a MySQL Cluster, as the cluster handles this automatically; however, it may be necessary in some situations to override the defaults provided by the cluster. See Section 17.3.2.8, “MySQL Cluster TCP/IP Connections”, for information about available TCP/IP configuration parameters and how to use them. (You may also find Section 17.3.2.9, “MySQL Cluster TCP/IP Connections Using Direct Connections” to be of interest in some cases.)

[shm]: Defines shared-memory connections between nodes. In MySQL 5.1, it is enabled by default, but should still be considered experimental. For a discussion of SHM interconnects, see Section 17.3.2.10, “MySQL Cluster Shared-Memory Connections”.

[sci]:Defines Scalable Coherent Interface connections between cluster data nodes. Such connections require software which, while freely available, is not part of the MySQL Cluster distribution, as well as specialized hardware. See Section 17.3.2.11, “SCI Transport Connections in MySQL Cluster” for detailed information about SCI interconnects.

MySQL Cluster software version

Numbers of data nodes and SQL nodes

Hardware

Operating system

Amount of data to be stored

Size and type of load under which the cluster is to operate

--ndb-use-exact-count=0

--ndb-index-stat-enable=0

--ndb-force-send=1

--engine-condition-pushdown=1

Each executable has its own command-line option which enables specifying the management server at startup. (See the documentation for the respective executable.)

It is also possible to set the connection string for all nodes in the cluster at once by placing it in a [mysql_cluster] section in the management server's my.cnf file.

For backward compatibility, two other options are available, using the same syntax:

However, these are now deprecated and should not be used for new installations.

Id

This is a unique identifier, used to refer to the host computer elsewhere in the configuration file.

HostName

This is the computer's hostname or IP address.

Id

Each node in the cluster has a unique identity. For a management node, this is represented by an integer value in the range 1 to 63 inclusive (previous to MySQL Cluster NDB 6.1.1), or in the range 1 to 255 inclusive (MySQL Cluster NDB 6.1.1 and later). This ID is used by all internal cluster messages for addressing the node, and so must be unique for each MySQL Cluster node, regardless of the type of node.

The use of the Id parameter for identifying management nodes is deprecated in favor of NodeId beginning with MySQL Cluster NDB 6.2.19, MySQL Cluster NDB 6.3.39, MySQL Cluster NDB 7.0.20, and MySQL Cluster NDB 7.1.9. Although Id continues to be supported for backward compatibility, it now generates a warning.

NodeId

Each node in the cluster has a unique identity. For a management node, this is represented by an integer value in the range 1 to 63 inclusive (previous to MySQL Cluster NDB 6.1.1), or in the range 1 to 255 inclusive (MySQL Cluster NDB 6.1.1 and later). This ID is used by all internal cluster messages for addressing the node, and so must be unique for each MySQL Cluster node, regardless of the type of node.

NodeId is the preferred parameter name to use when identifying management nodes beginning with MySQL Cluster NDB 6.2.19, MySQL Cluster NDB 6.3.39, MySQL Cluster NDB 7.0.20, and MySQL Cluster NDB 7.1.9. Although Id continues to be supported for backward compatibility, it is now deprecated and generates a warning when used.

ExecuteOnComputer

This refers to the Id set for one of the computers defined in a [computer] section of the config.ini file.

PortNumber

This is the port number on which the management server listens for configuration requests and management commands.

HostName

Specifying this parameter defines the hostname of the computer on which the management node is to reside. To specify a hostname other than localhost, either this parameter or ExecuteOnComputer is required.

LogDestination

This parameter specifies where to send cluster logging information. There are three options in this regard—CONSOLE, SYSLOG, and FILE—with FILE being the default:

It is possible to specify multiple log destinations separated by semicolons as shown here:

CONSOLE;SYSLOG:facility=local0;FILE:filename=/var/log/mgmd

ArbitrationRank

This parameter is used to define which nodes can act as arbitrators. Only management nodes and SQL nodes can be arbitrators. ArbitrationRank can take one of the following values:

Normally, the management server should be configured as an arbitrator by setting its ArbitrationRank to 1 (the default for management nodes) and those for all SQL nodes to 0 (the default for SQL nodes).

Beginning with MySQL 5.1.16 and MySQL Cluster NDB 6.1.3, it is possible to disable arbitration completely by setting ArbitrationRank to 0 on all management and SQL nodes. In MySQL Cluster NDB 7.0.7 and later releases, you can also control arbitration by overriding this parameter; to do this, set the Arbitration parameter in the [ndbd default] section of the config.ini global configuration file.

ArbitrationDelay

An integer value which causes the management server's responses to arbitration requests to be delayed by that number of milliseconds. By default, this value is 0; it is normally not necessary to change it.

DataDir

This specifies the directory where output files from the management server will be placed. These files include cluster log files, process output files, and the daemon's process ID (PID) file. (For log files, this location can be overridden by setting the FILE parameter for LogDestination as discussed previously in this section.)

The default value for this parameter is the directory in which ndb_mgmd is located.

PortNumberStats

This parameter specifies the port number used to obtain statistical information from a MySQL Cluster management server. It has no default value.

Wan