Table of Contents
This chapter contains information about MySQL
Cluster, which is a high-availability, high-redundancy
version of MySQL adapted for the distributed computing environment.
Current releases of MySQL Cluster use versions 6 and 7 of the
NDB
storage engine (also known as
NDBCLUSTER
) to enable running several
computers with MySQL servers and other software in a cluster.
Beginning with MySQL 5.1.24, support for the
NDBCLUSTER
storage engine was removed
from the standard MySQL server binaries built by MySQL. Instead,
users of MySQL Cluster binaries built by MySQL should upgrade to the
most recent binary release of MySQL Cluster NDB 7.0 or MySQL Cluster
7.1 for supported platforms—these include RPMs that should
work with most Linux distributions. MySQL Cluster users who build
from source should be aware that, also beginning with MySQL 5.1.24,
NDBCLUSTER
sources in the standard
MySQL 5.1 tree are no longer maintained; these users should use the
sources provided for MySQL Cluster NDB 7.0 or later. (Locations
where the sources can be obtained are listed later in this section.)
MySQL Cluster NDB 6.1, 6.2, and 6.3 were formerly known as “MySQL Cluster Carrier Grade Edition”. Beginning with MySQL Cluster NDB 6.2.15 and MySQL Cluster NDB 6.3.14, this term is no longer applied to the MySQL Cluster software—which is now known simply as “MySQL Cluster”—but rather to a commercial licensing and support package. You can learn more about available options for commercial licensing of MySQL Cluster from http://mysql.com/products/database/cluster/features.html, on the MySQL web site.
This chapter contains information about MySQL Cluster in MySQL 5.1 mainline releases through MySQL 5.1.23, MySQL Cluster NDB 6.2 releases through 5.1.51-ndb-6.2.19, MySQL Cluster NDB 6.3 releases through 5.1.73-ndb-6.3.55, MySQL Cluster NDB 7.0 releases through 5.1.73-ndb-7.0.42 and MySQL Cluster NDB 7.1 releases through 5.1.73-ndb-7.1.34. Currently, the MySQL Cluster NDB 7.2 release series is Generally Available (GA), as is MySQL Cluster NDB 7.1. MySQL Cluster NDB 7.0 and MySQL Cluster NDB 6.3 are previous GA release series; although they are still supported, we recommend that new deployments use MySQL Cluster NDB 7.2. For information about MySQL Cluster NDB 7.2, see MySQL Cluster NDB 7.2, in the MySQL 5.5 Manual.
This chapter also contains historical information about MySQL Cluster NDB 6.1 and MySQL Cluster NDB 6,2, although these release series are no longer in active development, and no longer supported for new deployments. You should upgrade to a MySQL Cluster NDB 7.1 or later release series as soon as possible.
Release notes for the changes in each release of MySQL Cluster are located at MySQL Cluster 7.1 Release Notes.
Supported Platforms. MySQL Cluster is currently available and supported on a number of platforms. For exact levels of support available for on specific combinations of operating system versions, operating system distributions, and hardware platforms, please refer to http://www.mysql.com/support/supportedplatforms/cluster.html.
Availability. MySQL Cluster binary and source packages are available for supported platforms from http://dev.mysql.com/downloads/cluster/.
Binary releases and RPMs were not available for MySQL Cluster NDB 6.2 prior to MySQL Cluster NDB 6.2.15.
MySQL Cluster release numbers.
Starting with MySQL Cluster NDB 6.1 and MySQL Cluster NDB 6.2,
MySQL Cluster follows a somewhat different release pattern from
the mainline MySQL 5.1 Cluster series of releases. In this
Manual and other MySQL documentation, we
identify these and later MySQL Cluster releases employing a
version number that begins with “NDB”. This version
number is that of the NDBCLUSTER
storage engine used in the release, and not of the MySQL server
version on which the MySQL Cluster release is based.
Version strings used in MySQL Cluster NDB 6.x and 7.x software. The version string displayed by MySQL Cluster NDB 6.x and 7.x software uses this format:
mysql-mysql_server_version
-ndb-ndb_engine_version
mysql_server_version
represents the
version of the MySQL Server on which the MySQL Cluster release is
based. For all MySQL Cluster NDB 6.x and 7.x releases, this is
“5.1”. ndb_engine_version
is
the version of the NDB
storage engine
used by this release of the MySQL Cluster software. You can see this
format used in the mysql client, as shown here:
shell>mysql
Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.1.73-ndb-7.1.34 Source distribution Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql>SELECT VERSION()\G
*************************** 1. row *************************** VERSION(): 5.1.73-ndb-7.1.34 1 row in set (0.00 sec)
This version string is also displayed in the output of the
SHOW
command in the ndb_mgm
client:
ndb_mgm> SHOW
Connected to Management Server at: localhost:1186
Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=1 @10.0.10.6 (5.1.73-ndb-7.1.34, Nodegroup: 0, *)
id=2 @10.0.10.8 (5.1.73-ndb-7.1.34, Nodegroup: 0)
[ndb_mgmd(MGM)] 1 node(s)
id=3 @10.0.10.2 (5.1.73-ndb-7.1.34)
[mysqld(API)] 2 node(s)
id=4 @10.0.10.10 (5.1.73-ndb-7.1.34)
id=5 (not connected, accepting connect from any host)
The version string identifies the mainline MySQL version from which
the MySQL Cluster release was branched and the version of the
NDBCLUSTER
storage engine used. For
example, the full version string for MySQL Cluster NDB 7.0.5 (the
first GA MySQL Cluster NDB 7.0 binary release) was
mysql-5.1.32-ndb-7.0.5
. From this we can
determine the following:
Since the portion of the version string preceding
“-ndb-
” is the base MySQL Server
version, this means that MySQL Cluster NDB 7.0.5 derives from
the MySQL 5.1.32, and contains all feature enhancements and
bugfixes from MySQL 5.1 up to and including MySQL 5.1.32.
Since the portion of the version string following
“-ndb-
” represents the version
number of the NDB
(or
NDBCLUSTER
) storage engine, MySQL
Cluster NDB 7.0.5 uses version 7.0.5 of the
NDBCLUSTER
storage engine.
New MySQL Cluster releases are numbered according to updates in the
NDB
storage engine, and do not necessarily
correspond in a linear fashion with mainline MySQL Server releases.
For example, MySQL Cluster NDB 7.0.5 (as previously noted) is based
on MySQL 5.1.32, and MySQL Cluster NDB 7.0.6 is based on MySQL
5.1.34 (version string: mysql-5.1.34-ndb-7.0.6
).
Compatibility with standard MySQL 5.1 releases.
While many standard MySQL schemas and applications can work using
MySQL Cluster, it is also true that unmodified applications and
database schemas may be slightly incompatible or have suboptimal
performance when run using MySQL Cluster (see
Section 17.1.6, “Known Limitations of MySQL Cluster”). Most of these issues
can be overcome, but this also means that you are very unlikely to
be able to switch an existing application datastore—that
currently uses, for example, MyISAM
or InnoDB
—to use the
NDB
storage engine without allowing
for the possibility of changes in schemas, queries, and
applications. Moreover, from MySQL 5.1.24 onwards, the MySQL
Server and MySQL Cluster codebases diverge considerably (and
NDB
storage engine support dropped
from subsequent MySQL Server releases), so that the standard
mysqld cannot function as a dropin replacement
for the version of mysqld that is supplied with
MySQL Cluster.
MySQL Cluster development source trees. MySQL Cluster development trees can also be accessed from https://code.launchpad.net/~mysql/:
The MySQL Cluster development sources maintained at https://code.launchpad.net/~mysql/ are licensed under the GPL. For information about obtaining MySQL sources using Bazaar and building them yourself, see Section 2.11.3, “Installing MySQL Using a Development Source Tree”.
Currently, MySQL Cluster NDB 7.0, MySQL Cluster NDB 7.1, and MySQL Cluster NDB 7.2 releases are all Generally Available (GA), although we recommend that new deployments use MySQL Cluster NDB 7.2. MySQL Cluster NDB 6.1, MySQL Cluster NDB 6.2, and MySQL Cluster NDB 6.3, are no longer in active development. For an overview of major features added in MySQL Cluster NDB 6.x and 7.x releases, see Section 17.1.4, “MySQL Cluster Development History”. For an overview of major features added in MySQL Cluster NDB 7.2, see MySQL Cluster Development History.
This chapter represents a work in progress, and its contents are subject to revision as MySQL Cluster continues to evolve. Additional information regarding MySQL Cluster can be found on the MySQL Web site at http://www.mysql.com/products/cluster/.
Additional Resources. More information about MySQL Cluster can be found in the following places:
For answers to some commonly asked questions about MySQL Cluster, see Section A.10, “MySQL 5.1 FAQ: MySQL Cluster”.
The MySQL Cluster mailing list: http://lists.mysql.com/cluster.
The MySQL Cluster Forum: http://forums.mysql.com/list.php?25.
Many MySQL Cluster users and developers blog about their experiences with MySQL Cluster, and make feeds of these available through PlanetMySQL.
MySQL Cluster is a technology that enables clustering of in-memory databases in a shared-nothing system. The shared-nothing architecture enables the system to work with very inexpensive hardware, and with a minimum of specific requirements for hardware or software.
MySQL Cluster is designed not to have any single point of failure. In a shared-nothing system, each component is expected to have its own memory and disk, and the use of shared storage mechanisms such as network shares, network file systems, and SANs is not recommended or supported.
MySQL Cluster integrates the standard MySQL server with an in-memory
clustered storage engine called NDB
(which stands for “Network
DataBase”). In our
documentation, the term NDB
refers to
the part of the setup that is specific to the storage engine,
whereas “MySQL Cluster” refers to the combination of
one or more MySQL servers with the NDB
storage engine.
A MySQL Cluster consists of a set of computers, known as hosts, each running one or more processes. These processes, known as nodes, may include MySQL servers (for access to NDB data), data nodes (for storage of the data), one or more management servers, and possibly other specialized data access programs. The relationship of these components in a MySQL Cluster is shown here:
All these programs work together to form a MySQL Cluster (see
Section 17.4, “MySQL Cluster Programs”. When data is stored by the
NDB
storage engine, the tables (and
table data) are stored in the data nodes. Such tables are directly
accessible from all other MySQL servers (SQL nodes) in the cluster.
Thus, in a payroll application storing data in a cluster, if one
application updates the salary of an employee, all other MySQL
servers that query this data can see this change immediately.
Although a MySQL Cluster SQL node uses the mysqld server daemon, it differs in a number of critical respects from the mysqld binary supplied with the MySQL 5.1 distributions, and the two versions of mysqld are not interchangeable.
In addition, a MySQL server that is not connected to a MySQL Cluster
cannot use the NDB
storage engine and
cannot access any MySQL Cluster data.
The data stored in the data nodes for MySQL Cluster can be mirrored; the cluster can handle failures of individual data nodes with no other impact than that a small number of transactions are aborted due to losing the transaction state. Because transactional applications are expected to handle transaction failure, this should not be a source of problems.
Individual nodes can be stopped and restarted, and can then rejoin the system (cluster). Rolling restarts (in which all nodes are restarted in turn) are used in making configuration changes and software upgrades (see Section 17.5.5, “Performing a Rolling Restart of a MySQL Cluster”). In MySQL Cluster NDB 7.0 and later, rolling restarts are also used as part of the process of adding new data nodes online (see Section 17.5.13, “Adding MySQL Cluster Data Nodes Online”). For more information about data nodes, how they are organized in a MySQL Cluster, and how they handle and store MySQL Cluster data, see Section 17.1.2, “MySQL Cluster Nodes, Node Groups, Replicas, and Partitions”.
Backing up and restoring MySQL Cluster databases can be done using
the NDB
-native functionality found in the MySQL
Cluster management client and the ndb_restore
program included in the MySQL Cluster distribution. For more
information, see Section 17.5.3, “Online Backup of MySQL Cluster”, and
Section 17.4.20, “ndb_restore — Restore a MySQL Cluster Backup”. You can also
use the standard MySQL functionality provided for this purpose in
mysqldump and the MySQL server. See
Section 4.5.4, “mysqldump — A Database Backup Program”, for more information.
MySQL Cluster nodes can use a number of different transport mechanisms for inter-node communications, including TCP/IP using standard 100 Mbps or faster Ethernet hardware. It is also possible to use the high-speed Scalable Coherent Interface (SCI) protocol with MySQL Cluster, although this is not required to use MySQL Cluster. SCI requires special hardware and software; see Section 17.3.5, “Using High-Speed Interconnects with MySQL Cluster”, for more about SCI and using it with MySQL Cluster.
NDBCLUSTER
(also known as NDB
) is an in-memory
storage engine offering high-availability and data-persistence
features.
The NDBCLUSTER
storage engine can be
configured with a range of failover and load-balancing options,
but it is easiest to start with the storage engine at the cluster
level. MySQL Cluster's NDB
storage
engine contains a complete set of data, dependent only on other
data within the cluster itself.
The “Cluster” portion of MySQL Cluster is configured independently of the MySQL servers. In a MySQL Cluster, each part of the cluster is considered to be a node.
In many contexts, the term “node” is used to indicate a computer, but when discussing MySQL Cluster it means a process. It is possible to run multiple nodes on a single computer; for a computer on which one or more cluster nodes are being run we use the term cluster host.
There are three types of cluster nodes, and in a minimal MySQL Cluster configuration, there will be at least three nodes, one of each of these types:
Management node: The role of this type of node is to manage the other nodes within the MySQL Cluster, performing such functions as providing configuration data, starting and stopping nodes, running backup, and so forth. Because this node type manages the configuration of the other nodes, a node of this type should be started first, before any other node. An MGM node is started with the command ndb_mgmd.
Data node: This type of node stores cluster data. There are as many data nodes as there are replicas, times the number of fragments (see Section 17.1.2, “MySQL Cluster Nodes, Node Groups, Replicas, and Partitions”). For example, with two replicas, each having two fragments, you need four data nodes. One replica is sufficient for data storage, but provides no redundancy; therefore, it is recommended to have 2 (or more) replicas to provide redundancy, and thus high availability. A data node is started with the command ndbd (see Section 17.4.1, “ndbd — The MySQL Cluster Data Node Daemon”). In MySQL Cluster NDB 7.0 and later, ndbmtd can also be used for the data node process; see Section 17.4.3, “ndbmtd — The MySQL Cluster Data Node Daemon (Multi-Threaded)”, for more information.
MySQL Cluster tables are normally stored completely in memory rather than on disk (this is why we refer to MySQL Cluster as an in-memory database). In MySQL 5.1, MySQL Cluster NDB 6.X, and later, some MySQL Cluster data can be stored on disk; see Section 17.5.12, “MySQL Cluster Disk Data Tables”, for more information.
SQL node: This is a node that accesses
the cluster data. In the case of MySQL Cluster, an SQL node is
a traditional MySQL server that uses the
NDBCLUSTER
storage engine. An SQL
node is a mysqld process started with the
--ndbcluster
and
--ndb-connectstring
options, which are
explained elsewhere in this chapter, possibly with additional
MySQL server options as well.
An SQL node is actually just a specialized type of API node, which designates any application which accesses MySQL Cluster data. Another example of an API node is the ndb_restore utility that is used to restore a cluster backup. It is possible to write such applications using the NDB API. For basic information about the NDB API, see Getting Started with the NDB API.
It is not realistic to expect to employ a three-node setup in a production environment. Such a configuration provides no redundancy; to benefit from MySQL Cluster's high-availability features, you must use multiple data and SQL nodes. The use of multiple management nodes is also highly recommended.
For a brief introduction to the relationships between nodes, node groups, replicas, and partitions in MySQL Cluster, see Section 17.1.2, “MySQL Cluster Nodes, Node Groups, Replicas, and Partitions”.
Configuration of a cluster involves configuring each individual node in the cluster and setting up individual communication links between nodes. MySQL Cluster is currently designed with the intention that data nodes are homogeneous in terms of processor power, memory space, and bandwidth. In addition, to provide a single point of configuration, all configuration data for the cluster as a whole is located in one configuration file.
The management server manages the cluster configuration file and the cluster log. Each node in the cluster retrieves the configuration data from the management server, and so requires a way to determine where the management server resides. When interesting events occur in the data nodes, the nodes transfer information about these events to the management server, which then writes the information to the cluster log.
In addition, there can be any number of cluster client processes
or applications. These include standard MySQL clients,
NDB
-specific API programs, and management
clients. These are described in the next few paragraphs.
Standard MySQL clients. MySQL Cluster can be used with existing MySQL applications written in PHP, Perl, C, C++, Java, Python, Ruby, and so on. Such client applications send SQL statements to and receive responses from MySQL servers acting as MySQL Cluster SQL nodes in much the same way that they interact with standalone MySQL servers.
MySQL clients using a MySQL Cluster as a data source can be
modified to take advantage of the ability to connect with multiple
MySQL servers to achieve load balancing and failover. For example,
Java clients using Connector/J 5.0.6 and later can use
jdbc:mysql:loadbalance://
URLs (improved in
Connector/J 5.1.7) to achieve load balancing transparently; for
more information about using Connector/J with MySQL Cluster, see
Using Connector/J with MySQL Cluster.
NDB client programs.
Client programs can be written that access MySQL Cluster data
directly from the NDBCLUSTER
storage engine,
bypassing any MySQL Servers that may connected to the cluster,
using the NDB API, a high-level C++ API.
Such applications may be useful for specialized purposes where
an SQL interface to the data is not needed. For more
information, see The NDB API.
Beginning with MySQL Cluster NDB 7.1,
NDB
-specific Java applications can also be
written for MySQL Cluster, using the MySQL Cluster
Connector for Java. This MySQL Cluster Connector
includes ClusterJ, a high-level database
API similar to object-relational mapping persistence frameworks
such as Hibernate and JPA that connect directly to
NDBCLUSTER
, and so does not require access to a
MySQL Server. Support is also provided in MySQL Cluster NDB 7.1
and later for ClusterJPA, an OpenJPA
implementation for MySQL Cluster that leverages the strengths of
ClusterJ and JDBC; ID lookups and other fast operations are
performed using ClusterJ (bypassing the MySQL Server), while more
complex queries that can benefit from MySQL's query optimizer
are sent through the MySQL Server, using JDBC. See
Java and MySQL Cluster, and
The ClusterJ API and Data Object Model, for more
information.
Management clients. These clients connect to the management server and provide commands for starting and stopping nodes gracefully, starting and stopping message tracing (debug versions only), showing node versions and status, starting and stopping backups, and so on. An example of this type of program is the ndb_mgm management client supplied with MySQL Cluster (see Section 17.4.5, “ndb_mgm — The MySQL Cluster Management Client”). Such applications can be written using the MGM API, a C-language API that communicates directly with one or more MySQL Cluster management servers. For more information, see The MGM API.
Oracle also makes available MySQL Cluster Manager, which provides an advanced command-line interface simplifying many complex MySQL Cluster management tasks, such restarting a MySQL Cluster with a large number of nodes. The MySQL Cluster Manager client also supports commands for getting and setting the values of most node configuration parameters as well as mysqld server options and variables relating to MySQL Cluster. See MySQL™ Cluster Manager 1.3.2 User Manual, for more information.
Event logs. MySQL Cluster logs events by category (startup, shutdown, errors, checkpoints, and so on), priority, and severity. A complete listing of all reportable events may be found in Section 17.5.6, “Event Reports Generated in MySQL Cluster”. Event logs are of the two types listed here:
Cluster log: Keeps a record of all desired reportable events for the cluster as a whole.
Node log: A separate log which is also kept for each individual node.
Under normal circumstances, it is necessary and sufficient to keep and examine only the cluster log. The node logs need be consulted only for application development and debugging purposes.
Checkpoint.
Generally speaking, when data is saved to disk, it is said that
a checkpoint has been reached. More
specific to MySQL Cluster, a checkpoint is a point in time where
all committed transactions are stored on disk. With regard to
the NDB
storage engine, there are
two types of checkpoints which work together to ensure that a
consistent view of the cluster's data is maintained. These are
shown in the following list:
Local Checkpoint (LCP): This is a checkpoint that is specific to a single node; however, LCPs take place for all nodes in the cluster more or less concurrently. An LCP involves saving all of a node's data to disk, and so usually occurs every few minutes. The precise interval varies, and depends upon the amount of data stored by the node, the level of cluster activity, and other factors.
Global Checkpoint (GCP): A GCP occurs every few seconds, when transactions for all nodes are synchronized and the redo-log is flushed to disk.
For more information about the files and directories created by local checkpoints and global checkpoints, see MySQL Cluster Data Node File System Directory Files.
This section discusses the manner in which MySQL Cluster divides and duplicates data for storage.
A number of concepts central to an understanding of this topic are discussed in the next few paragraphs.
(Data) Node. An ndbd process, which stores a replica —that is, a copy of the partition (see below) assigned to the node group of which the node is a member.
Each data node should be located on a separate computer. While it is also possible to host multiple ndbd processes on a single computer, such a configuration is not supported.
It is common for the terms “node” and “data node” to be used interchangeably when referring to an ndbd process; where mentioned, management nodes (ndb_mgmd processes) and SQL nodes (mysqld processes) are specified as such in this discussion.
Node Group. A node group consists of one or more nodes, and stores partitions, or sets of replicas (see next item).
The number of node groups in a MySQL Cluster is not directly
configurable; it is a function of the number of data nodes and of
the number of replicas
(NoOfReplicas
configuration parameter), as shown here:
[number_of_node_groups
] =number_of_data_nodes
/NoOfReplicas
Thus, a MySQL Cluster with 4 data nodes has 4 node groups if
NoOfReplicas
is set to 1
in the config.ini
file, 2 node groups if
NoOfReplicas
is set to 2,
and 1 node group if
NoOfReplicas
is set to 4.
Replicas are discussed later in this section; for more information
about NoOfReplicas
, see
Section 17.3.2.6, “Defining MySQL Cluster Data Nodes”.
All node groups in a MySQL Cluster must have the same number of data nodes.
Prior to MySQL Cluster NDB 7.0, it was not possible to add new data nodes to a MySQL Cluster without shutting down the cluster completely and reloading all of its data. In MySQL Cluster NDB 7.0 (beginning with MySQL Cluster version NDB 6.4.0), you can add new node groups (and thus new data nodes) to a running MySQL Cluster—see Section 17.5.13, “Adding MySQL Cluster Data Nodes Online”, for information about how this can be done.
Partition. This is a portion of the data stored by the cluster. There are as many cluster partitions as nodes participating in the cluster. Each node is responsible for keeping at least one copy of any partitions assigned to it (that is, at least one replica) available to the cluster.
A replica belongs entirely to a single node; a node can (and usually does) store several replicas.
NDB and user-defined partitioning.
MySQL Cluster normally partitions
NDBCLUSTER
tables automatically.
However, in MySQL 5.1 and later MySQL Cluster releases, it is
possible to employ user-defined partitioning with
NDBCLUSTER
tables. This is subject
to the following limitations:
Only KEY
and LINEAR KEY
partitioning schemes can be used with
NDB
tables.
When using ndbd, the maximum number of
partitions that may be defined explicitly for any
NDB
table is 8 *
[
.
(The number of node groups in a MySQL Cluster is determined as
discussed previously in this section.)
number of node groups
]
When using ndbmtd, this maximum is also
affected by the number of local query handler threads, which
is determined by the value of the
MaxNoOfExecutionThreads
configuration parameter. In such cases, the maximum number of
partitions that may be defined explicitly for an
NDB
table is equal to 4
* MaxNoOfExecutionThreads * [
.
number of node
groups
]
See Section 17.4.3, “ndbmtd — The MySQL Cluster Data Node Daemon (Multi-Threaded)”, for more information.
For more information relating to MySQL Cluster and user-defined partitioning, see Section 17.1.6, “Known Limitations of MySQL Cluster”, and Section 18.5.2, “Partitioning Limitations Relating to Storage Engines”.
Replica. This is a copy of a cluster partition. Each node in a node group stores a replica. Also sometimes known as a partition replica. The number of replicas is equal to the number of nodes per node group.
The following diagram illustrates a MySQL Cluster with four data nodes, arranged in two node groups of two nodes each; nodes 1 and 2 belong to node group 0, and nodes 3 and 4 belong to node group 1. Note that only data (ndbd) nodes are shown here; although a working cluster requires an ndb_mgm process for cluster management and at least one SQL node to access the data stored by the cluster, these have been omitted in the figure for clarity.
The data stored by the cluster is divided into four partitions, numbered 0, 1, 2, and 3. Each partition is stored—in multiple copies—on the same node group. Partitions are stored on alternate node groups as follows:
Partition 0 is stored on node group 0; a primary replica (primary copy) is stored on node 1, and a backup replica (backup copy of the partition) is stored on node 2.
Partition 1 is stored on the other node group (node group 1); this partition's primary replica is on node 3, and its backup replica is on node 4.
Partition 2 is stored on node group 0. However, the placing of its two replicas is reversed from that of Partition 0; for Partition 2, the primary replica is stored on node 2, and the backup on node 1.
Partition 3 is stored on node group 1, and the placement of its two replicas are reversed from those of partition 1. That is, its primary replica is located on node 4, with the backup on node 3.
What this means regarding the continued operation of a MySQL Cluster is this: so long as each node group participating in the cluster has at least one node operating, the cluster has a complete copy of all data and remains viable. This is illustrated in the next diagram.
In this example, where the cluster consists of two node groups of two nodes each, any combination of at least one node in node group 0 and at least one node in node group 1 is sufficient to keep the cluster “alive” (indicated by arrows in the diagram). However, if both nodes from either node group fail, the remaining two nodes are not sufficient (shown by the arrows marked out with an X); in either case, the cluster has lost an entire partition and so can no longer provide access to a complete set of all cluster data.
One of the strengths of MySQL Cluster is that it can be run on commodity hardware and has no unusual requirements in this regard, other than for large amounts of RAM, due to the fact that all live data storage is done in memory. (It is possible to reduce this requirement using Disk Data tables—see Section 17.5.12, “MySQL Cluster Disk Data Tables”, for more information about these.) Naturally, multiple and faster CPUs can enhance performance. Memory requirements for other MySQL Cluster processes are relatively small.
The software requirements for MySQL Cluster are also modest. Host operating systems do not require any unusual modules, services, applications, or configuration to support MySQL Cluster. For supported operating systems, a standard installation should be sufficient. The MySQL software requirements are simple: all that is needed is a production release of MySQL Cluster. It is not strictly necessary to compile MySQL yourself merely to be able to use MySQL Cluster. We assume that you are using the binaries appropriate to your platform, available from the MySQL Cluster software downloads page at http://dev.mysql.com/downloads/cluster/.
For communication between nodes, MySQL Cluster supports TCP/IP networking in any standard topology, and the minimum expected for each host is a standard 100 Mbps Ethernet card, plus a switch, hub, or router to provide network connectivity for the cluster as a whole. We strongly recommend that a MySQL Cluster be run on its own subnet which is not shared with machines not forming part of the cluster for the following reasons:
Security. Communications between MySQL Cluster nodes are not encrypted or shielded in any way. The only means of protecting transmissions within a MySQL Cluster is to run your MySQL Cluster on a protected network. If you intend to use MySQL Cluster for Web applications, the cluster should definitely reside behind your firewall and not in your network's De-Militarized Zone (DMZ) or elsewhere.
See Section 17.5.11.1, “MySQL Cluster Security and Networking Issues”, for more information.
Efficiency. Setting up a MySQL Cluster on a private or protected network enables the cluster to make exclusive use of bandwidth between cluster hosts. Using a separate switch for your MySQL Cluster not only helps protect against unauthorized access to MySQL Cluster data, it also ensures that MySQL Cluster nodes are shielded from interference caused by transmissions between other computers on the network. For enhanced reliability, you can use dual switches and dual cards to remove the network as a single point of failure; many device drivers support failover for such communication links.
Network communication and latency. MySQL Cluster requires communication between data nodes and API nodes (including SQL nodes), as well as between data nodes and other data nodes, to execute queries and updates. Communication latency between these processes can directly affect the observed performance and latency of user queries. In addition, to maintain consistency and service despite the silent failure of nodes, MySQL Cluster uses heartbeating and timeout mechanisms which treat an extended loss of communication from a node as node failure. This can lead to reduced redundancy. Recall that, to maintain data consistency, a MySQL Cluster shuts down when the last node in a node group fails. Thus, to avoid increasing the risk of a forced shutdown, breaks in communication between nodes should be avoided wherever possible.
The failure of a data or API node results in the abort of all uncommitted transactions involving the failed node. Data node recovery requires synchronization of the failed node's data from a surviving data node, and re-establishment of disk-based redo and checkpoint logs, before the data node returns to service. This recovery can take some time, during which the Cluster operates with reduced redundancy.
Heartbeating relies on timely generation of heartbeat signals by all nodes. This may not be possible if the node is overloaded, has insufficient machine CPU due to sharing with other programs, or is experiencing delays due to swapping. If heartbeat generation is sufficiently delayed, other nodes treat the node that is slow to respond as failed.
This treatment of a slow node as a failed one may or may not be
desirable in some circumstances, depending on the impact of the
node's slowed operation on the rest of the cluster. When
setting timeout values such as
HeartbeatIntervalDbDb
and
HeartbeatIntervalDbApi
for
MySQL Cluster, care must be taken care to achieve quick detection,
failover, and return to service, while avoiding potentially
expensive false positives.
Where communication latencies between data nodes are expected to be higher than would be expected in a LAN environment (on the order of 100 µs), timeout parameters must be increased to ensure that any allowed periods of latency periods are well within configured timeouts. Increasing timeouts in this way has a corresponding effect on the worst-case time to detect failure and therefore time to service recovery.
LAN environments can typically be configured with stable low latency, and such that they can provide redundancy with fast failover. Individual link failures can be recovered from with minimal and controlled latency visible at the TCP level (where MySQL Cluster normally operates). WAN environments may offer a range of latencies, as well as redundancy with slower failover times. Individual link failures may require route changes to propagate before end-to-end connectivity is restored. At the TCP level this can appear as large latencies on individual channels. The worst-case observed TCP latency in these scenarios is related to the worst-case time for the IP layer to reroute around the failures.
SCI support. It is also possible to use the high-speed Scalable Coherent Interface (SCI) with MySQL Cluster, but this is not a requirement. See Section 17.3.5, “Using High-Speed Interconnects with MySQL Cluster”, for more about this protocol and its use with MySQL Cluster.
In this section, we discuss changes in the implementation of MySQL Cluster in MySQL 5.1, MySQL Cluster NDB 6.x, and MySQL Cluster NDB 7.x, as compared to earlier MySQL Cluster releases.
There are a number of significant changes in the implementation of
the NDBCLUSTER
storage engine in
mainline MySQL 5.1 releases up to and including MySQL 5.1.23 as
compared to that in MySQL 5.0; MySQL Cluster NDB 6.x and 7.x make
further changes and improvements in MySQL Cluster in addition to
these. The changes and features most likely to be of interest are
shown in the following tables:
MySQL Cluster NDB 7.2 |
---|
Distribution of MySQL users and privileges across MySQL Cluster SQL nodes |
Distributed pushed-down joins, with greatly improved performance for many joins that can thus be pushed down and executed in parallel on the data nodes. |
Improved default values for data node configuration parameters. |
MySQL Cluster NDB 7.1 |
---|
Production-level support for MySQL Cluster on Microsoft Windows platforms. |
ndbinfo meta-information database |
MySQL Cluster Connector for Java, including ClusterJ and OpenJPA (ClusterJPA) support |
Native support for default column values |
MySQL Cluster NDB 7.0 |
---|
Multi-threaded data nodes (ndbmtd data node daemon) |
Online addition of data nodes; online data redistribution |
MySQL on Windows (alpha; source releases only) |
Configuration cache |
Backup snapshots (START BACKUP ... SNAPSHOTSTART ,
START BACKUP ... SNAPSHOTEND commands) |
IPv6 support for geo-replication |
Protected DDL operations |
Dynamic buffering for NDB transporters |
Increased flexibility in determining arbitration handling, using a new
Arbitration data
node configuration parameter |
NDB API counters and associated status variables, which make it possible
to observe the API-level effects of SQL statements on
NDB tables from the
mysql client. |
MySQL Cluster NDB 6.3 |
---|
Conflict detection and resolution for multi-master replication |
Compressed backups and local checkpoints |
Support for OPTIMIZE TABLE |
Parallel data node recovery |
Enhanced transaction coordinator selection |
Improved SQL statement performance metrics |
Transaction batching |
ndb_restore attribute promotion |
Support for epoll (Linux only) |
Distribution awareness |
NDB thread locks; realtime extensions for
multiple CPUs |
MySQL Cluster NDB 6.2 |
---|
Improved backup status reporting
(BackupReportFrequency ,
REPORT BackupStatus ) |
Multiple connections per SQL node |
Data access with NdbRecord (NDB API) |
REPORT MemoryUsage command |
Memory allocation improvements |
Management client connection control |
Micro-GCPs |
Online ADD COLUMN ; improved online index creation |
MySQL Cluster NDB 6.1 |
---|
Greater number of cluster nodes |
Disabling of arbitration |
Additional DUMP commands |
Faster Disk Data backups |
Batched slave updates |
MySQL 5.1 (through 5.1.23) |
---|
MySQL Cluster Replication |
Disk Data storage |
Variable-size columns |
User-defined partitioning |
Autodiscovery of table schema changes |
Online adding and dropping of indexes |
The following improvements to MySQL Cluster were made in MySQL Cluster NDB 7.2.0.
Distribution of MySQL users and privileges.
Automatic distribution of MySQL users and privileges
across all SQL nodes in a given MySQL Cluster is now
supported. To enable this support, you must first import
an SQL script
share/mysql/ndb_dist_priv.sql
that is
included with the MySQL Cluster NDB 7.2 distribution. This
script creates several stored procedures which you can use
to enable privilege distribution and perform related
tasks.
When a new MySQL Server joins a MySQL Cluster where privilege distribution is in effect, it also participates in the privilege distribution automatically.
Once privilege distribution is enabled, all changes to the
grant tables made on any mysqld attached
to the cluster are immediately available on any other
attached MySQL Servers. This is true whether the changes are
made using CREATE USER
,
GRANT
, or any of the other
statements described elsewhere in this Manual (see
Section 13.7.1, “Account Management Statements”.) This includes
privileges relating to stored routines and views; however,
automatic distribution of the views or stored routines
themselves is not currently supported.
For more information, see Section 17.5.14, “Distributed MySQL Privileges for MySQL Cluster”.
Distributed pushed-down joins.
Many joins can now be pushed down to the NDB kernel for
processing on MySQL Cluster data nodes. Previously, a join
was handled in MySQL Cluster by means of repeated accesses
of NDB
by the SQL node;
however, when pushed-down joins are enabled, a pushable
join is sent in its entirety to the data nodes, where it
can be distributed among the data nodes and executed in
parallel on multiple copies of the data, with a single,
merged result being returned to mysqld.
This can reduce greatly the number of round trips between
an SQL node and the data nodes required to handle such a
join, leading to greatly improved performance of join
processing.
It is possible to determine when joins can be pushed down to
the data nodes by examining the join with
EXPLAIN
. A number of new
system status variables
(Ndb_pushed_queries_defined
,
Ndb_pushed_queries_dropped
,
Ndb_pushed_queries_executed
,
and Ndb_pushed_reads
) and
additions to the counters
table (in the ndbinfo
information database) can also be helpful in determining
when and how well joins are being pushed down.
More information and examples are available in the
description of the
ndb_join_pushdown
server
system variable. See also the description of the status
variables referenced in the previous paragraph, as well as
Section 17.5.10.7, “The ndbinfo counters Table”.
Improved default values for data node configuration parameters.
In order to provide more resiliency to environmental
issues and better handling of some potential failure
scenarios, and to perform more reliably with increases in
memory and other resource requirements brought about by
recent improvements in join handling by
NDB
, the default values for a
number of MySQL Cluster data node configuration parameters
have been changed. The parameters and changes are
described in the following list:
HeartbeatIntervalDbDb
:
Default increased from 1500 ms to 5000 ms.
ArbitrationTimeout
:
Default increased from 3000 ms to 7500 ms.
TimeBetweenEpochsTimeout
:
Now effectively disabled by default (default changed
from 4000 ms to 0).
SharedGlobalMemory
:
Default increased from 20 MB to 128 MB.
MaxParallelScansPerFragment
:
Default increased from 32 to 256.
In addition, the value computed for
MaxNoOfLocalScans
when this parameter is not set in
config.ini
has been increased by a
factor of 4.
MySQL Cluster NDB 7.2.1 and later MySQL Cluster NDB 7.2 releases are based on MySQL Server 5.5 and are not covered here. See MySQL Cluster NDB 7.2.
The following improvements to MySQL Cluster have been made in MySQL Cluster NDB 7.1.
MySQL Cluster information database (ndbinfo).
The ndbinfo
information
database makes it possible to obtain real-time
characteristics of a MySQL Cluster by issuing queries from
the mysql client or other MySQL client
applications. ndbinfo
provides metadata specific to MySQL Cluster similarly to
how the INFORMATION_SCHEMA
database
provides metadata for the standard MySQL Server. This
eliminates much of the need to read log files, issue
REPORT
or DUMP
commands in the ndb_mgm client, or
parse the output of ndb_config in order
to get configuration and status information from a running
MySQL Cluster.
For more information, see Section 17.5.10, “The ndbinfo MySQL Cluster Information Database”.
Java connectors for MySQL Cluster.
The MySQL Cluster distribution now includes 2 new Java
user APIs, ClusterJ and ClusterJPA. ClusterJ is an
object-relational interface in a manner similar to that of
Java persistence frameworks such as Hibernate. Cluster JPA
is a reimplementation of OpenJPA. ClusterJ uses a backend
library (NdbJTie) that provides access to the
NDB
storage engine without
using a MySQL Server connection or JDBC. ClusterJPA also
uses NdbJTie when it improves performance, but can also
process complex queries using JDBC and a MySQL Server
connection, where it can take advantage of the MySQL query
optimizer.
ClusterJ and Cluster JPA can also be made to work with recent MySQL Cluster NDB 7.0 releases although the necessary library and JAR files are included only in MySQL Cluster NDB 7.1.1 and later.
For more information about using ClusterJ and ClusterJPA, see MySQL Cluster Connector for Java.
New CHANGE MASTER TO option for circular replication.
Beginning with MySQL Cluster NDB 7.1.0, the
CHANGE MASTER TO
statement
supports an IGNORE_SERVER_IDS
option
which takes a comma-separated list of server IDs and
causes events originating from the corresponding servers
to be ignored. (Log rotation and log deletion events are
preserved.)
See Section 13.4.2.1, “CHANGE MASTER TO Syntax”, as well as Section 13.7.5.36, “SHOW SLAVE STATUS Syntax”, for more information.
Native support for default column values.
Starting with MySQL Cluster NDB 7.1.4, default values for
table columns are stored in the NDB kernel rather than by
the MySQL server as was done previously. This means that
inserts on tables having column value defaults can be
smaller and faster than before, because less data must be
sent from SQL nodes to NDBCLUSTER
.
Tables created using previous MySQL Cluster releases can
still be used in MySQL Cluster 7.1.4 and later; however,
they do not support native default values until they are
upgraded. You can upgrade a table with non-native default
values to support native default values using an offline
ALTER TABLE
statement.
MySQL Cluster on Windows (Production). Beginning with MySQL Cluster NDB 7.1.3, MySQL Cluster is available for production use on Microsoft Windows operating systems; MySQL Cluster NDB 7.1 binaries for Windows can be obtained from cluster/.
Features and behavior are generally comparable to those found on previously supported platforms such as Linux and Solaris. However, you must install the binaries manually.
Beginning with MySQL Cluster NDB 7.1.5, MySQL Cluster processes can be run as Windows services.
If you wish to build MySQL Cluster from source on Windows,
you must configure the build using the
WITH_NDBCLUSTER_STORAGE_ENGINE
option.
For more information, see
Section 2.11.7, “Installing MySQL from Source on Windows”.
--nowait-nodes option for management servers.
It is now possible to configure a cluster with two
management servers, but to start the cluster using only
one of them by starting the management node daemon with
the --nowait-nodes
option. The other management server can then be started at
a later time to join the running MySQL Cluster.
Improved lock handling for primary key lookups on BLOB tables.
A MySQL Cluster table stores all but the first 256 bytes
of any BLOB
or
TEXT
column values in a
separate BLOB
table; when
executing queries against such tables, a shared lock is
obtained. Prior to MySQL Cluster NDB 7.1.1, when the query
used a primary key lookup and took place within a
transaction, the lock was held for the duration of the
transaction, even after no more data was being read from
the NDB
table. Now in such
cases, the lock is released when all
BLOB
data associated with
the table has been read. (Bug #49190)
A shared lock is also taken for unique key lookups; it is still the case that this lock is held for the duration of the transaction.
Heartbeat thread policy and priority.
Beginning with MySQL Cluster NDB 7.1.2, a new
configuration parameter
HeartbeatThreadPriority
makes it
possible to set the policy and the priority for the
heartbeat thread on management and API nodes.
Improved access to partitioning information.
The ndb_desc utility now provides
additional information about the partitioning of data
stored in MySQL Cluster. Beginning with MySQL Cluster NDB
7.1.2, the --blob-info
option causes this program to include partition
information for BLOB
tables
in its output. Also beginning with MySQL Cluster NDB
7.1.2, the
--extra-node-info
option
causes ndb_desc to include information
about data distribution (that is, which table fragments
are stored on which data nodes). Each of these options
also requires the use of the
--extra-partition-info
option.
Information about partition-to-node mappings can also be
obtained using the
Table::getFragmentNodes()
method, also added in MySQL Cluster NDB 7.1.2.
Replication attribute promotion and demotion.
Beginning with MySQL Cluster NDB 7.1.3, MySQL Cluster
Replication supports attribute promotion and demotion when
replicating between columns of different but similar types
on the master and the slave. For example, it is possible
to promote an INT
column on
the master to a BIGINT
column on the slave, and to demote a
TEXT
column to a
VARCHAR
column.
The implementation of type demotion distinguishes between
lossy and non-lossy type conversions, and their use on the
slave can be controlled by setting the
slave_type_conversions
global server system variable.
For more information, see Attribute promotion and demotion (MySQL Cluster).
Change in ndbinfo database.
The experimental pools
table was removed from
ndbinfo
in MySQL Cluster
NDB 7.1.3. Applications which used this table can and
should be rewritten to use other
ndbinfo
tables.
Configuration caching control.
Beginning with MySQL Cluster NDB 7.1.4, it is possible to
disable the management server's configuration cache
using the --config-cache
option, which forces ndb_mgmd to read
its configuration data from the
config.ini
configuration file every
time it starts. For more information about configuration
caching and this option, see
Section 17.3.2, “MySQL Cluster Configuration Files”. See also
Section 17.4.4, “ndb_mgmd — The MySQL Cluster Management Server Daemon”.
Incompatible change in NDB API event reporting.
Beginning with MySQL Cluster NDB 7.1.4, DDL events are no
longer reported on Event
objects by default. Instead the desired level of event
reporting
(EventReport
value)
must be set explicitly using
Event::setReport()
.
Number of table attributes. Beginning with MySQL Cluster NDB 7.1.4, the maximum number of attributes (columns plus indexes) per table has been increased from 128 to 512.
InnoDB support in commercial binaries.
Beginning with MySQL Cluster NDB 7.1.4b, all commercial
binary releases of MySQL Cluster provide support for the
InnoDB
storage engine.
Heartbeat ordering.
Beginning with MySQL Cluster NDB 7.1.5, it is possible to
set a specific order for transmission of heartbeats
between data nodes, using the
HeartbeatOrder
data node configuration parameter introduced in this
version. This parameter can be useful in situations where
multiple data nodes are running on the same host and a
temporary disruption in connectivity between hosts would
otherwise cause the loss of a node group (and thus failure
of the cluster).
Relaxed ndb_restore column comparison rules. When restoring data, ndb_restore compares the attributes of a column for equality with the definition of the column in the target table. However, not all of these attributes need to be the same for ndb_restore to be meaningful, safe and useful. Beginning with MySQL Cluster NDB 7.1.5, ndb_restore automatically ignores differences in certain column attributes which do not necessarily have to match between the version of the column in a backup and the version of that column in the MySQL Cluster to which the column data is being restored. These attributes include the following:
COLUMN_FORMAT
setting
(FIXED
, DYNAMIC
,
or DEFAULT
)
STORAGE
setting
(MEMORY
or DISK
)
The default value
The distribution key
In such cases, ndb_restore reports any such differences to minimize any chance of user error.
Storage of user data in anyValue.
When writing NDB events to the binary log, MySQL Cluster
uses
OperationOptions::anyValue
to store the server ID. Beginning with MySQL Cluster NDB
7.1.6, it is possible to store user data from an NDB API
application in part of the anyValue
when mysqld has been started with the
--server-id-bits
option set
to a nondefault value. Also beginning with MySQL Cluster
NDB 7.1.6, it is possible to view this data in the output
of mysqlbinlog, for which its own
--server-id-bits
option is added.
--add-drop-trigger option for mysqldump.
Beginning with MySQL Cluster NDB 7.1.8, this option can be
used to force all CREATE
TRIGGER
statements in
mysqldump output to be preceded by a
DROP TRIGGER
IF EXISTS
statement.
Forcing node shutdown and restart.
In MySQL Cluster NDB 7.1.8 and later, it is possible using
the ndb_mgm management client or the
MGM API to force a data node shutdown or restart even if
this would force the shutdown or restart of the entire
cluster. In the management client, this is implemented
through the addition of the -f
(force)
option to the STOP
and
RESTART
commands. For more information,
see Section 17.5.2, “Commands in the MySQL Cluster Management Client”.
The MGM API also adds two new methods for forcing such a
node shutdown or restart; see
ndb_mgm_stop4(), and
ndb_mgm_restart4(), for more
information about these methods.
Disk Data usage statistics (diskpagebuffer table).
MySQL Cluster 7.1.9 introduces a new table in the
ndbinfo
information
database. The
diskpagebuffer
table
provides real-time data on disk page buffer usage. These
statistics can be used to monitor performance of MySQL
Cluster Disk Data read and write operations, and can prove
useful in the tuning of Disk Data parameters such as
DiskPageBufferMemory
.
InnoDB Plugin support.
Beginning with MySQL Cluster NDB 7.1.9, the MySQL Server
supplied with MySQL Cluster supports the
InnoDB
Plugin. For more
information about enabling the plugin if you are building
MySQL Cluster from source, see
Section 17.2, “MySQL Cluster Installation and Upgrades”.
Due to a packaging issue, the
InnoDB
plugin was not
included in RPMs for MySQL Cluster NDB 7.1.9; this issue
was corrected in MySQL Cluster NDB 7.1.9a. (Bug #58283)
TimeBetweenEpochsTimeout and GCP stop control.
Beginning with MySQL Cluster NDB 7.1.10, it is possible to
disable GCP stops by setting
TimeBetweenEpochsTimeout
to 0. In addition, a warning is written to the cluster log
whenever the time required for a GCP save exceeds 60
seconds or the time required for a GCP commit exceeds 10
seconds. This warning includes a report of the current
value of
TimeBetweenEpochsTimeout
.
For more information, see
Disk Data and GCP Stop errors.
Skipping corrupted tables in NDB native backups.
Beginning with MySQL Cluster NDB 7.1.10, you can cause
ndb_restore to ignore tables that are
corrupted due to missing blob parts tables by using the
--skip-broken-objects
option. When this option is used, such tables are skipped,
and the restoration of any remaining uncorrupted tables in
the backup continues.
BLOB read and write batching.
Beginning with MySQL Cluster NDB 7.1.10, it possible to
control batching of BLOB
read and write operations. For SQL nodes, this can be done
using the
--ndb-blob-read-batch-bytes
and
--ndb-blob-write-batch-bytes
options for mysqld. In NDB API
applications, you can control batching of
BLOB
reads and writes using
the NdbTransaction
methods
setMaxPendingBlobReadBytes()
,
getMaxPendingBlobReadBytes()
,
setMaxPendingBlobWriteBytes()
,
and
getMaxPendingBlobWriteBytes()
.
Restoring from a NDB native backup to a differently-named database.
MySQL Cluster NDB 7.1.11 adds a new
--rewrite-database
option to ndb_restore, which makes it
possible to restore to a database having a different name
from that of the database in the backup. The option can be
used multiple times, and it is possible to restore from
more than one source database in the backup to a single
target database (although no protection against table or
other object name collision is provided).
See Section 17.4.20, “ndb_restore — Restore a MySQL Cluster Backup”, for more information.
Selective over-commit handling.
Beginning with MySQL Cluster NDB 7.1.10, it is possible to
exercise more direct control over uncommitted operations
from transactions aborted due to timeouts flushing redo
logs to disk. This is implemented using three
configuration parameters added in this version of MySQL
Cluster: the data node configuration parameters
RedoOverCommitCounter
and
RedoOverCommitLimit
,
and the API node configuration parameter
DefaultOperationRedoProblemAction
.
When an attempt to flush a given redo log takes longer than
RedoOverCommitLimit
seconds, and this occurs
RedoOverCommitLimit
times or more, the transactions contained within the redo
log are aborted. Any operations left uncommitted as a result
are either aborted or re-tried, according to the value of
DefaultOperationRedoProblemAction
.
For more information, see Redo log over-commit handling
INFORMATION_SCHEMA improvements.
Beginning with MySQL Cluster NDB 7.1.11, Beginning with
MySQL Cluster NDB 7.0.22,
INFORMATION_SCHEMA
provides disk usage
information for MySQL Cluster Disk Data tables.
Previously,
INFORMATION_SCHEMA.TABLES
showed only the space usage for the in-memory data part of
the table. Now, it also shows the space allocated for and
used by the disk_part data of that table as well.
In addition, the
INFORMATION_SCHEMA.PARTITIONS
table (which did not show any statistics for
NDB
tables) now shows correct
values in this table's TABLE_ROWS
,
AVG_ROW_LENGTH
,
DATA_LENGTH
,
MAX_DATA_LENGTH
, and
DATA_FREE
columns, for each partition.
ndb_restore attribute demotion.
Beginning with MySQL Cluster NDB 7.1.11, it is possible to
enable attribute demotion when restoring a MySQL Cluster
from a native backup running
ndb_restore with a new
--lossy-conversions
option.
In general,the rules governing demotion are the same as for MySQL replication, although there are some exceptions that you may need to take into account being employing this option. See Section 16.4.1.9.2, “Replication of Columns Having Different Data Types”, for information about type conversions currently supported by attribute promotion and demotion in MySQL Cluster.
Improved multi-threaded order index building.
Previously, it was not possible to enable multi-threaded
building of ordered indexes during initial restarts. In
MySQL Cluster NDB 7.1.11, this can now be done, using the
new
TwoPassInitialNodeRestartCopy
data node configuration parameter.
Configuration version information in ndbinfo.nodes.
You can see which version or versions of the MySQL Cluster
configuration file are in effect on the data nodes by
checking the config_generation
column
which is added to the
nodes
table in MySQL
Cluster NDB 7.1.13.
Configuration version information in ndbinfo.nodes.
You can see which version or versions of the MySQL Cluster
configuration file are in effect on the data nodes by
checking the config_generation
column
which is added to the
nodes
table in MySQL
Cluster NDB 7.1.13.
Improvements in adding data nodes online.
Begining with MySQL Cluster NDB 7.0.24, it is possible to
add data nodes online to a running MySQL Cluster without
performing a rolling restart of the cluster or starting
data node processes with the
--nowait-nodes
option. This
can be done by setting
Nodegroup = 65536
in the config.ini
file for any data
nodes that should be started at a later time, when first
starting the cluster. The amount of time the cluster waits
before doing this can be controlled using the
StartNoNodeGroupTimeout
data node configuration parameter.
Unique key updates in replication.
It is possible in MySQL Cluster NDB 7.1.14 and later to
employ operations that update unique keys when replicating
NDB
tables. Previously this
could lead to duplicate key errors when trying to execute
the binary log (due to the fact that row events in the
binary log were ordered according to the partitioning of
the base table, and could differ in order within the epoch
for that in which they were executed).
Master and slave tables must both be using the
NDB
storage engine for this
to work.
Starting with MySQL Cluster NDB 7.1.17, the NDB kernel
implements a number of statistical counters relating to
actions performed by or affecting
Ndb
objects. Such events
include starting, closing, and aborting transactions;
operations using primary keys or unique keys; table, range,
and pruned scans; blocking of threads by incompleted
operations; and data and events sent and received by MySQL
Cluster nodes. NDB API statistics counters are incremented
inside the NDB kernel whenever NDB API calls are made or
data is sent to or received by the data nodes. A MySQL
Server running as an SQL node in the cluster can access the
values of these counters as system status variables, as seen
in the output of SHOW STATUS
,
or in the results of queries against the
SESSION_STATUS
or
GLOBAL_STATUS
table in the INFORMATION_SCHEMA
database.
By comparing the values of these status variables before and
after the execution of statements affecting
NDB
tables, you can observe the
corresponding actions taken on the NDB API level. This can
be highly useful when monitoring and tuning MySQL Cluster.
For more information, see Section 17.5.15, “NDB API Statistics Counters and Variables”. See also Section 17.3.4.4, “MySQL Cluster Status Variables”, for information about the individual status variables.
Version 2 binary log row events.
New versions of the WRITE_ROW
,
UPDATE_ROW
, and
DELETE_ROW
events have been implemented
in MySQL Cluster NDB 7.1.16, extending them with
additional information intended to support future
enhancements; these are referred to as “Version
2” binary log events.
Version 2 log events are not backward compatible, and cannot
be read by older slaves. A new mysqld
option
--log-bin-use-v1-row-events
can be employed to force use of Version 1 events when
writing the binary log. This can be used during upgrades to
make a newer mysqld generate Version 1
binary log row events that can be read by older slaves.
Circular replication: Primary wins first conflict detection.
Two new conflict detection functions
NDB$EPOCH()
and
NDB$EPOCH_TRANS()
can be useful in
“active-active” circular replication
scenarios with two MySQL Clusters. For each case, we
designate one cluster as primary and one as secondary, and
implement a “primary always wins” rule for
determining whether to accept conflicting changes. When
using NDB$EPOCH()
, conflicting rows on
the secondary are realigned with those on the primary;
when using NDB$EPOCH_TRANS()
, it is
transactions containing rows in conflict (and any
transactions which depend on them) on the secondary that
are realigned.
When using NDB$EPOCH_TRANS()
as the
conflict detection function, the binary log must be written
using Version 2 binary logging row events; that is, the
mysqld processes on both the primary and
the secondary must be started with
--log-bin-use-v1-row-events=0
.
In addition, the secondary's binary log must include
transaction IDs for all rows, that is, by setting
--ndb-log-transaction-id=1
.
You can monitor conflict detection and resolution performed
using these functions by reading a number of related server
status variables, including
Ndb_conflict_fn_epoch
,
Ndb_conflict_fn_epoch_trans
,
and
Ndb_conflict_trans_row_conflict_count
(among others). See
Section 17.3.4.4, “MySQL Cluster Status Variables”.
For more information, see Section 17.6.11, “MySQL Cluster Replication Conflict Resolution”.
Fail-fast data node capability.
Beginning with MySQL Cluster NDB 7.1.17, it is possible to
enable fail-fast behavior for data nodes by enabling the
CrashOnCorruptedTuple
configuration parameter introduced in this version
(disabled by default). Doing so causes data nodes to fail
whenever they detect a corrupted tuple.
Rows per partition limit removed. Previously it was possible to store a maximum of 46137488 rows in a single MySQL Cluster partition—that is, per data node. Beginning with MySQL Cluster NDB 7.0.36 and MySQL Cluster NDB 7.1.25, this limitation has been lifted, and there is no longer any practical upper limit to this number. (Bug #13844405, Bug #14000373)
MySQL Cluster NDB 7.1 is also supported by MySQL Cluster Manager, which provides an advanced command-line interface that can simplify many complex MySQL Cluster management tasks. See MySQL™ Cluster Manager 1.3.2 User Manual, for more information.
The following list provides an overview of significant feature additions and changes made in MySQL Cluster NDB 7.0. For more detailed information about all feature changes and bugfixes made in MySQL Cluster NDB 7.0, see http://dev.mysql.com/doc/relnotes/mysql-cluster/7.1/en/mysql-cluster-news-7-0.html.
Early development versions of MySQL Cluster NDB 7.0 were known as “MySQL Cluster NDB 6.4”, and the first four releases in this series were identified as MySQL Cluster NDB 6.4.0 through 6.4.3. Any information relating to these MySQL Cluster NDB 6.4.x releases appearing in this documentation apply to MySQL Cluster NDB 7.0.
MySQL Cluster NDB 7.0.4 is the fifth MySQL Cluster NDB 7.0 release; it is the successor to MySQL Cluster NDB 6.4.3.
MySQL Cluster on Windows (alpha).
MySQL Cluster NDB 7.0 is available on an experimental
basis for Windows operating systems (for production use on
Windows, you should use MySQL Cluster NDB 7.1.3 or later).
Features and behavior comparable to those found on
platforms that are already supported—such as Linux
and Solaris—are planned for MySQL Cluster on
Windows. In MySQL Cluster NDB 7.0, you must build from
source (Windows binaries are available for MySQL Cluster
NDB 7.1 releases). To enable MySQL Cluster support on
Windows when building from source, you must configure the
build using the
WITH_NDBCLUSTER_STORAGE_ENGINE
option.
For more information, see
Section 2.11.7, “Installing MySQL from Source on Windows”.
Ability to add nodes and node groups online.
Beginning with MySQL Cluster NDB 6.4.0, it is possible to
add new node groups (and thus new data nodes) to a running
MySQL Cluster without shutting down and reloading the
cluster. As part of enabling this feature, a new command
CREATE NODEGROUP
has been added to the
cluster management client and the functionality of the
ALTER ONLINE
TABLE ... REORGANIZE PARTITION
SQL statement has
been extended. For more information, see
Section 17.5.13, “Adding MySQL Cluster Data Nodes Online”.
Data node multi-threading support.
Beginning with MySQL Cluster NDB 6.4.0, a multi-threaded
version of the data node daemon, named
ndbmtd, is available for use on data
node hosts with multiple CPU cores. This binary is built
automatically when compiling with MySQL Cluster support;
no additional options other than those needed to provide
MySQL Cluster support are needed when configuring the
build. In most respects, ndbmtd
functions in the same way as ndbd, and
can use the same command-line options and configuration
parameters. In addition, the new
MaxNoOfExecutionThreads
configuration parameter can be used to determine the
number of data node process threads for
ndbmtd. For more information, see
Section 17.4.3, “ndbmtd — The MySQL Cluster Data Node Daemon (Multi-Threaded)”.
Disk Data tables are not yet supported for use with ndbmtd.
Configuration cache.
Formerly, MySQL Cluster configuration was
stateless—that is, configuration information was
reloaded from the cluster's global configuration file
(usually config.ini
) each time
ndb_mgmd was started. Beginning with
MySQL Cluster NDB 6.4.0, the cluster's configuration
is cached internally, and the global configuration file is
no longer automatically re-read when the management server
is restarted. This behavior can be controlled using the
management server options --configdir
,
--initial
, and --reload
.
In MySQL Cluster NDB 7.0.15 and later, the configuration
cache can be disabled using the
--config-cache
option. For more
information about these changes, see
Section 17.3.2, “MySQL Cluster Configuration Files”. For more
information about the new management server options, see
Section 17.4.4, “ndb_mgmd — The MySQL Cluster Management Server Daemon”.
Detection of NDB API client connection errors.
In MySQL Cluster NDB 7.0 (6.4.0 and later releases), the
NDB API's
Ndb_cluster_connection
class adds the
get_latest_error()
and
get_latest_error_msg()
methods for catching and diagnosing problems with NDB API
client connections.
Snapshot options for backups.
Beginning with MySQL Cluster NDB 6.4.0, you can determine
when performing a cluster backup whether the backup
matches the state of the data when the backup was started
or when it was completed, using the new options
SNAPSHOTSTART
and
SNAPSHOTEND
for the management
client's START BACKUP
command. See
Section 17.5.3.2, “Using The MySQL Cluster Management Client to Create a Backup”,
for more information.
Dynamic NDB transporter send buffer memory allocation.
Previously, the NDB kernel used a fixed-size send buffer
for every data node in the cluster, which was allocated
when the node started. Because the size of this buffer
could not be changed after the cluster was started, it was
necessary to make it large enough in advance to
accommodate the maximum possible load on any transporter
socket. However, this was an inefficient use of memory,
since much of it often went unused. Beginning with MySQL
Cluster NDB 6.4.0, send buffer memory is allocated
dynamically from a memory pool shared between all
transporters, which means that the size of the send buffer
can be adjusted as necessary. This change is reflected by
the addition of the configuration parameters
TotalSendBufferMemory
,
ReservedSendBufferMemory
,
and OverLoadLimit
, as well as a change
in how the existing
SendBufferMemory
configuration parameter is used. For more information, see
Section 17.3.2.13, “Configuring MySQL Cluster Send Buffer Parameters”.
Robust DDL operations.
Beginning with MySQL Cluster NDB 6.4.0, DDL operations
(such as CREATE TABLE
or
ALTER TABLE
) are protected
from data node failures; in the event of a data node
failure, such operations are now rolled back gracefully.
Previously, if a data node failed while trying to perform
a DDL operation, the MySQL Cluster data dictionary became
locked and no further DDL statements could be executed
without restarting the cluster.
IPv6 support in MySQL Cluster Replication. Beginning with MySQL Cluster NDB 6.4.1, IPv6 networking is supported between MySQL Cluster SQL nodes, which makes it possible to replicate between instances of MySQL Cluster using IPv6 addresses. However, IPv6 is supported only for direct connections between MySQL servers; all connections within an individual MySQL Cluster must use IPv4. For more information, see Section 17.6.3, “Known Issues in MySQL Cluster Replication”.
Restoring specific databases, tables, or columns from a MySQL Cluster
backup.
It is now possible to exercise more fine-grained control
when restoring a MySQL Cluster from backup using
ndb_restore. Beginning with MySQL
Cluster NDB 6.4.3, you can choose to restore only
specified tables or databases, or exclude specific tables
or databases from being restored, using the new
ndb_restore options
--include-tables
,
--include-databases
,
--exclude-tables
, and
--exclude-databases
. Beginning with MySQL
Cluster NDB 7.0.7, it is also possible to restore to a
table having fewer columns than the original using the
--exclude-missing-columns
option. For
more information about all of these options, see
Section 17.4.20, “ndb_restore — Restore a MySQL Cluster Backup”.
Improved Disk Data file system configuration.
As of MySQL Cluster NDB 6.4.3, you can specify default
locations for MySQL Cluster Disk Data data files and undo
log files using the data node configuration parameters
FileSystemPathDD
,
FileSystemPathDataFiles
,
and
FileSystemPathUndoFiles
.
This eliminates the need to use symbolic links to place
Disk Data files separately from other files in data node
file systems to improve Disk Data performance. For more
information, see
Disk Data file system parameters.
Automatic creation of Disk Data log file groups and tablespaces.
Beginning with MySQL Cluster NDB 6.4.3, using the data
node configuration parameters
InitialLogFileGroup
and
InitialTablespace
,
you can cause the creation of a MySQL Cluster Disk Data
log file group, tablespace, or both, when the cluster is
first started. When using these parameters, no SQL
statements are required to create these Disk Data objects.
For more information, see
Disk
Data object creation parameters.
Improved internal message passing and record handling. MySQL Cluster NDB 7.0 contains 2 changes that optimize the use of network connections by addressing the size and number of messages passed between data nodes, and between data nodes and API nodes, which can increase MySQL Cluster and application performance:
Packed reads. Formerly, each read request signal contained a list of columns to be retrieved, each of these column identifiers using 4 bytes within the message. This meant that the message size increased as the number of columns being fetched increased. In addition, in the response from the data node, each column result was packed to a 4-byte boundary, which resulted in wasted space. In MySQL Cluster NDB 7.0, messaging for read operations is optimized in both directions, using a bitmap in the read request to specify the columns to be fetched. Where many fields are requested, this can result in a significant message size reduction as compared with the old method. In addition, the 4-byte packing in responses is no longer used, which means that smaller fields consume less space.
Long signal transactions. This enhancement reduces the number of messages and signals that are sent to data nodes for complex requests. Prior to MySQL Cluster NDB 7.0, there was a 100 byte limit on the size of the request signal, which meant that complex requests had to be split up between multiple messages prior to transmission, then reassembled on the receiving end. In addition to actual payload data, each message required its own operating system and protocol overhead such as header information. This often wasted network bandwidth and data node CPU. The maximum size of the message is now 32 KB, which is sufficient to accommodate most queries.
Both of these optimizations are internal to the NDB API, and so is transparent to applications; this is true whether an application uses the NDB API directly or does so indirectly through an SQL node.
Configuration parameter data dumps.
Starting with MySQL Cluster NDB 7.0.6, the
ndb_config utility supports a
--configinfo
option that causes it to
dump a list of all configuration parameters supported by
the cluster, along with brief descriptions, information
about the parameters' default and permitted values,
and the sections of the config.ini
file in which the parameters apply. An additional
--xml
switch causes
ndb_config to use XML rather than
plaintext output. Using ndb_config
--configinfo
or
ndb_config --configinfo
--xml
requires no access to a running
MySQL Cluster, any other programs, or any files. For more
information and examples, see
Section 17.4.7, “ndb_config — Extract MySQL Cluster Configuration Information”.
Per-table reporting of free space on disk.
The INFORMATION_SCHEMA.FILES
table shows information about used and free space in MySQL
Cluster Disk Data data files, but this information is not
applicable to individual tables. In MySQL Cluster NDB
7.0.8 and later, the ndb_desc utility
provides two additional columns in its output that show
the amount of space allocated on disk for a given
NDB
table as well the amount
of space that remains available for additional storage of
disk-based column data for that table. For more
information, see
Section 17.4.10, “ndb_desc — Describe NDB Tables”.
Improved restart times. Optimizations in redo log handling and other file system operations introduced in MySQL Cluster NDB 7.0.9 have the potential to reduce considerably the time required for restarts. While actual performance benefits observed in production setups will naturally vary depending on database size, hardware, and other conditions, our own preliminary testing has shown that these improvements can yield startup times that are faster than those typical of previous MySQL Cluster NDB 7.0 releases by a factor of 50 or more.
Native support for default column values.
Starting with MySQL Cluster NDB 7.0.15, default values for
table columns are stored in the NDB kernel rather than by
the MySQL server as was done previously. This means that
inserts on tables having column value defaults can be
smaller and faster than before, because less data must be
sent from SQL nodes to NDBCLUSTER
.
Tables created using previous MySQL Cluster releases can
still be used in MySQL Cluster 7.0.15 and later; however,
they do not support native default values until they are
upgraded. You can upgrade a table with non-native default
values to support native default values using an offline
ALTER TABLE
statement.
--nowait-nodes option for management servers.
Starting with MySQL Cluster NDB 7.0.10, it is possible to
configure a cluster with two management servers, but to
start the cluster using only one of them by starting the
management node daemon with the
--nowait-nodes
option.
The other management server can then be started at a later
time to join the running MySQL Cluster.
Increased flexibility in online upgrade procedure. Previously, when performing an upgrade of a running MySQL cluster, the order in which the types of cluster nodes had to be upgraded was very strict. However, beginning with MySQL Cluster NDB 7.0.10, MySQL Cluster supports online upgrading of API nodes (including MySQL servers running as SQL nodes) online upgrading management nodes, data nodes, or both.
Before attempting to use this new upgrade functionality, see Section 17.5.5, “Performing a Rolling Restart of a MySQL Cluster”, for additional information, especially if you are planning an online upgrade to MySQL Cluster NDB 7.0 from MySQL Cluster NDB 6.3.
New CHANGE MASTER TO option for circular replication.
Beginning with MySQL Cluster NDB 7.0.11, the
CHANGE MASTER TO
statement
supports an IGNORE_SERVER_IDS
option
which takes a comma-separated list of server IDs and
causes events originating from the corresponding servers
to be ignored. (Log rotation and log deletion events are
preserved.)
See Section 13.4.2.1, “CHANGE MASTER TO Syntax”, as well as Section 13.7.5.36, “SHOW SLAVE STATUS Syntax”, for more information.
New replication conflict resolution strategy.
Beginning with MySQL Cluster NDB 7.0.11, the function
NDB$MAX_DELETE_WIN()
is available to
implement “greatest timestamp, delete wins”
conflict resolution. See
NDB$MAX_DELETE_WIN(column_name),
for more information.
Improved lock handling for primary key lookups on BLOB tables.
A MySQL Cluster table stores all but the first 256 bytes
of any BLOB
or
TEXT
column values in a
separate BLOB
table; when
executing queries against such tables, a shared lock is
obtained. Prior to MySQL Cluster NDB 7.0.12, when the
query used a primary key lookup and took place within a
transaction, the lock was held for the duration of the
transaction, even after no more data was being read from
the NDB
table. Now in such
cases, the lock is released when all
BLOB
data associated with
the table has been read. (Bug #49190)
A shared lock is also taken for unique key lookups; it is still the case that this lock is held for the duration of the transaction.
Heartbeat thread policy and priority.
Beginning with MySQL Cluster NDB 7.0.13, a new
configuration parameter
HeartbeatThreadPriority
makes it
possible to set the policy and the priority for the
heartbeat thread on management and API nodes.
Improved access to partitioning information.
The ndb_desc utility now provides
additional information about the partitioning of data
stored in MySQL Cluster. Beginning with MySQL Cluster NDB
7.0.13, the --blob-info
option causes this program to include partition
information for BLOB
tables
in its output. Beginning with MySQL Cluster NDB 7.0.14,
the --extra-node-info
option causes ndb_desc to include
information about data distribution (that is, which table
fragments are stored on which data nodes). Each of these
options also requires the use of the
--extra-partition-info
option.
Information about partition-to-node mappings can also be
obtained using the
Table::getFragmentNodes()
method, also added in MySQL Cluster NDB 7.0.14.
Replication attribute promotion and demotion.
Beginning with MySQL Cluster NDB 7.0.14, MySQL Cluster
Replication supports attribute promotion and demotion when
replicating between columns of different but similar types
on the master and the slave. For example, it is possible
to promote an INT
column on
the master to a BIGINT
column on the slave, and to demote a
TEXT
column to a
VARCHAR
column.
The implementation of type demotion distinguishes between
lossy and non-lossy type conversions, and their use on the
slave can be controlled by setting the
slave_type_conversions
global server system variable.
For more information, see Attribute promotion and demotion (MySQL Cluster).
Incompatible change in NDB API event reporting.
Beginning with MySQL Cluster NDB 7.0.15, DDL events are no
longer reported on Event
objects by default. Instead such event reporting must be
enabled explicitly using the
Event::setReport()
method. For more information, see
Event::setReport(), and
The Event::EventReport Type.
Number of table attributes. Beginning with MySQL Cluster NDB 7.0.15, the maximum number of attributes (columns plus indexes) per table has been increased from 128 to 512.
Heartbeat ordering.
Beginning with MySQL Cluster NDB 7.0.16, it is possible to
set a specific order for transmission of heartbeats
between data nodes, using the
HeartbeatOrder
data node configuration parameter introduced in this
version. This parameter can be useful in situations where
multiple data nodes are running on the same host and a
temporary disruption in connectivity between hosts would
otherwise cause the loss of a node group (and thus failure
of the cluster).
Relaxed ndb_restore column comparison rules. When restoring data, ndb_restore compares the attributes of a column for equality with the definition of the column in the target table. However, not all of these attributes need to be the same for ndb_restore to be meaningful, safe and useful. Beginning with MySQL Cluster NDB 7.0.16, ndb_restore automatically ignores differences in certain column attributes which do not necessarily have to match between the version of the column in a backup and the version of that column in the MySQL Cluster to which the column data is being restored. These attributes include the following:
COLUMN_FORMAT
setting
(FIXED
, DYNAMIC
,
or DEFAULT
)
STORAGE
setting
(MEMORY
or DISK
)
The default value
The distribution key
In such cases, ndb_restore reports any such differences to minimize any chance of user error.
Storage of user data in anyValue.
When writing NDB events to the binary log, MySQL Cluster
uses
OperationOptions::anyValue
to store the server ID. Beginning with MySQL Cluster NDB
7.0.17, it is possible to store user data from an NDB API
application in part of the anyValue
when mysqld has been started with the
--server-id-bits
option set
to a nondefault value. Also beginning with MySQL Cluster
NDB 7.0.17, it is possible to view this data in the output
of mysqlbinlog, for which its own
--server-id-bits
option is added.
--add-drop-trigger option for mysqldump.
Beginning with MySQL Cluster NDB 7.0.19, this option can
be used to force all CREATE
TRIGGER
statements in
mysqldump output to be preceded by a
DROP TRIGGER
IF EXISTS
statement.
Forcing node shutdown and restart.
In MySQL Cluster NDB 7.0.19 and later, it is possible
using the ndb_mgm management client or
the MGM API to force a data node shutdown or restart even
if this would force the shutdown or restart of the entire
cluster. In the management client, this is implemented
through the addition of the -f
(force)
option to the STOP
and
RESTART
commands. For more information,
see Section 17.5.2, “Commands in the MySQL Cluster Management Client”.
The MGM API also adds two new methods for forcing such a
node shutdown or restart; see
ndb_mgm_stop4(), and
ndb_mgm_restart4(), for more
information about these methods.
TimeBetweenEpochsTimeout and GCP stop control.
Beginning with MySQL Cluster NDB 7.0.21, it is possible to
disable GCP stops by setting
TimeBetweenEpochsTimeout
to 0. In addition, a warning is written to the cluster log
whenever the time required for a GCP save exceeds 60
seconds or the time required for a GCP commit exceeds 10
seconds. This warning includes a report of the current
value of
TimeBetweenEpochsTimeout
.
For more information, see
Disk Data and GCP Stop errors.
Skipping corrupted tables in NDB native backups.
Beginning with MySQL Cluster NDB 7.0.21, you can cause
ndb_restore to ignore tables that are
corrupted due to missing blob parts tables by using the
--skip-broken-objects
option. When this option is used, such tables are skipped,
and the restoration of any remaining uncorrupted tables in
the backup continues.
BLOB read and write batching.
Beginning with MySQL Cluster NDB 7.0.21, it possible to
control batching of BLOB
read and write operations. For SQL nodes, this can be done
using the
--ndb-blob-read-batch-bytes
and
--ndb-blob-write-batch-bytes
options for mysqld. In NDB API
applications, you can control batching of
BLOB
reads and writes using
the NdbTransaction
methods
setMaxPendingBlobReadBytes()
,
getMaxPendingBlobReadBytes()
,
setMaxPendingBlobWriteBytes()
,
and
getMaxPendingBlobWriteBytes()
.
Restoring from a NDB native backup to a differently-named database.
MySQL Cluster NDB 7.0.22 adds a new
--rewrite-database
option to ndb_restore, which makes it
possible to restore to a database having a different name
from that of the database in the backup. The option can be
used multiple times, and it is possible to restore from
more than one source database in the backup to a single
target database (although no protection against table or
other object name collision is provided).
See Section 17.4.20, “ndb_restore — Restore a MySQL Cluster Backup”, for more information.
INFORMATION_SCHEMA inprovements.
Beginning with MySQL Cluster NDB 7.0.22,
INFORMATION_SCHEMA
provides disk usage
information for MySQL Cluster Disk Data tables.
Previously,
INFORMATION_SCHEMA.TABLES
showed only the space usage for the in-memory data part of
the table. Now, it also shows the space allocated for and
used by the disk_part data of that table as well.
In addition, the
INFORMATION_SCHEMA.PARTITIONS
table (which did not show any statistics for
NDB
tables) now shows correct
values in this table's TABLE_ROWS
,
AVG_ROW_LENGTH
,
DATA_LENGTH
,
MAX_DATA_LENGTH
, and
DATA_FREE
columns, for each partition.
Configuration version information in ndbinfo.nodes.
You can see which version or versions of the MySQL Cluster
configuration file are in effect on the data nodes by
checking the config_generation
column
which is added to the
nodes
table in MySQL
Cluster NDB 7.0.24.
Improvements in adding data nodes online.
Beginning with MySQL Cluster NDB 7.0.24, it is possible to
add data nodes online to a running MySQL Cluster without
performing a rolling restart of the cluster or starting
data node processes with the
--nowait-nodes
option. This
can be done by setting
Nodegroup = 65536
in the config.ini
file for any data
nodes that should be started at a later time, when first
starting the cluster. The amount of time the cluster waits
before doing this can be controlled using the
StartNoNodeGroupTimeout
data node configuration parameter.
Unique key updates in replication.
It is possible in MySQL Cluster NDB 7.0.25 and later to
employ operations that update unique keys when replicating
NDB
tables. Previously this
could lead to duplicate key errors when trying to execute
the binary log (due to the fact that row events in the
binary log were ordered according to the partitioning of
the base table, and could differ in order within the epoch
for that in which they were executed).
Master and slave tables must both be using the
NDB
storage engine for this
to work.
Starting with MySQL Cluster NDB 7.0.22, the NDB kernel
implements a number of statistical counters relating to
actions performed by or affecting
Ndb
objects. Such events
include starting, closing, and aborting transactions;
operations using primary keys or unique keys; table, range,
and pruned scans; blocking of threads by incompleted
operations; and data and events sent and received by MySQL
Cluster nodes. NDB API statistics counters are incremented
inside the NDB kernel whenever NDB API calls are made or
data is sent to or received by the data nodes. A MySQL
Server running as an SQL node in the cluster can access the
values of these counters as system status variables, as seen
in the output of SHOW STATUS
,
or in the results of queries against the
SESSION_STATUS
or
GLOBAL_STATUS
table in the INFORMATION_SCHEMA
database.
By comparing the values of these status variables before and
after the execution of statements affecting
NDB
tables, you can observe the
corresponding actions taken on the NDB API level. This can
be highly useful when monitoring and tuning MySQL Cluster.
For more information, see Section 17.5.15, “NDB API Statistics Counters and Variables”. See also Section 17.3.4.4, “MySQL Cluster Status Variables”, for information about the individual status variables.
Version 2 binary log row events.
New versions of the WRITE_ROW
,
UPDATE_ROW
, and
DELETE_ROW
events have been implemented
in MySQL Cluster NDB 7.0.27, extending them with
additional information intended to support future
enhancements; these are referred to as “Version
2” binary log events.
Version 2 log events are not backward compatible, and cannot
be read by older slaves. A new mysqld
option
--log-bin-use-v1-row-events
can be employed to force use of Version 1 events when
writing the binary log. This can be used during upgrades to
make a newer mysqld generate Version 1
binary log row events that can be read by older slaves.
Fail-fast data node capability.
Beginning with MySQL Cluster NDB 7.0.28, it is possible to
enable fail-fast behavior for data nodes by enabling the
CrashOnCorruptedTuple
configuration parameter introduced in this version
(disabled by default). Doing so causes data nodes to fail
whenever they detect a corrupted tuple.
Rows per partition limit removed. Previously it was possible to store a maximum of 46137488 rows in a single MySQL Cluster partition—that is, per data node. Beginning with MySQL Cluster NDB 7.0.36, this limitation has been lifted, and there is no longer any practical upper limit to this number.
The following list provides an overview of significant feature additions and changes first made in MySQL Cluster NDB 6.3. For more detailed information about all feature changes and bugfixes made in MySQL Cluster NDB 6.3, see http://dev.mysql.com/doc/relnotes/mysql-cluster/7.1/en/mysql-cluster-news-6-3.html.
Conflict detection and resolution. It is now possible to detect and resolve conflicts that arise in multi-master replication scenarios, such as circular replication, when different masters may try to update the same row on the slave with different data. Both “greatest timestamp wins” and “same timestamp wins” scenarios are supported. For more information, see Section 17.6.11, “MySQL Cluster Replication Conflict Resolution”.
Recovery of one master, many slaves replication setups.
Recovery of multi-way replication setups (“one
master, many slaves”) is now supported using the
--ndb-log-orig
server option and changes
in the mysql.ndb_binlog_index
table.
See Section 17.6.4, “MySQL Cluster Replication Schema and Tables”,
for more information.
Enhanced selection options for transaction coordinator.
New values and behaviors are introduced for
--ndb_optimized_node_selection
, enabling
greater flexibility when an SQL node chooses a transaction
coordinator. For more information, see the description of
ndb_optimized_node_selection
in Section 17.3.4.3, “MySQL Cluster System Variables”.
Replication heartbeats.
Replication heartbeats facilitate the task of monitoring
and detecting failures in master-slave connections in real
time. This feature is implemented using a new
MASTER_HEARTBEAT_PERIOD =
clause for the
value
CHANGE MASTER TO
statement
and the addition of two status variables
Slave_heartbeat_period
and
Slave_received_heartbeats
.
For more information, see
Section 13.4.2.1, “CHANGE MASTER TO Syntax”.
NDB thread locks.
It is possible to lock NDB
execution threads and maintenance threads (such as file
system and other operating system threads) to specific
CPUs on multiprocessor data node hosts, and to leverage
real-time scheduling.
Improved performance of updates using primary keys or unique keys. The number of unnecessary reads when performing a primary key or unique key update has been greatly reduced. Since it is seldom necessary to read a record prior to an update, this can yield a considerable improvement in performance. In addition, primary key columns are no longer written to when not needed during update operations.
Batching improvements.
Support of batched DELETE
and UPDATE
operations has
been significantly improved. Batching of UPDATE
WHERE...
and multiple
DELETE
operations is also
now implemented.
Improved SQL statement performance metrics.
The Ndb_execute_count
system status variable measures the number of round trips
made by SQL statements to the
NDB
kernel, providing an
improved metric for determining efficiency with which
statements are executed. For more information, see
MySQL
Cluster Status Variables:
Ndb_execute_count
.
Compressed LCPs and backups.
Compressed local checkpoints and backups can save 50% or
more of the disk space used by uncompressed LCPs and
backups. These can be enabled using the two new data node
configuration parameters
CompressedLCP
and
CompressedBackup
,
respectively.
OPTIMIZE TABLE support with NDBCLUSTER tables.
OPTIMIZE TABLE
is supported
for dynamic columns of in-memory
NDB
tables. In such cases, it
is no longer necessary to drop (and possibly to re-create)
a table, or to perform a rolling restart, in order to
recover memory from deleted rows for general re-use by
Cluster. The performance of OPTIMIZE
on
Cluster tables can be tuned by adjusting the value of the
ndb_optimization_delay
system variable,
which controls the number of milliseconds to wait between
processing batches of rows by
OPTIMIZE TABLE
. In
addition, OPTIMIZE TABLE
on
an NDBCLUSTER
table can be
interrupted by, for example, killing the SQL thread
performing the OPTIMIZE
operation.
Batching of transactions.
It is possible to cause statements occurring within the
same transaction to be run as a batch by setting the
session variable
transaction_allow_batching
to 1
or ON
. To use
this feature, autocommit
must be set to 0
or
OFF
. Batch sizes can be controlled
using the --ndb-batch-size
option for
mysqld. For additional information, see
Section 17.3.4.2, “MySQL Server Options for MySQL Cluster”,
and Section 17.3.4.3, “MySQL Cluster System Variables”.
Attribute promotion with ndb_restore.
It is possible using ndb_restore to
restore data reliably from a column of a given type to a
column that uses a “larger” type. This is
sometimes referred to as attribute
promotion. For example, MySQL Cluster backup
data that originated in a
SMALLINT
column can be
restored to a MEDIUMINT
,
INT
, or
BIGINT
column. See
Section 17.4.20, “ndb_restore — Restore a MySQL Cluster Backup”, for
more information.
Parallel data node recovery. Recovery of multiple data nodes can now be done in parallel, rather than sequentially. In other words, several data nodes can be restored concurrently, which can often result in much faster recovery times than when they are restored one at a time.
Increased local checkpoint efficiency. Only 2 local checkpoints are stored, rather than 3, lowering disk space requirements and the size and number of redo log files.
NDBCLUSTER table persistence control.
Persistence of NDB
tables can
be controlled using the session variables
ndb_table_temporary
and
ndb_table_no_logging
.
ndb_table_no_logging
causes
NDB
tables not to be
checkpointed to disk;
ndb_table_temporary
does the same, and
in addition, no schema files are created. See
Section 17.3.4.1, “MySQL Cluster mysqld Option and Variable Reference”.
Epoll support (Linux only). Epoll is an improved method for handling file descriptors, which is more efficient than scanning to determine whether a file descriptor has data to be read. (The term epoll is specific to Linux and equivalent functionality is known by other names on other platforms such as Solaris and FreeBSD.) Currently, MySQL Cluster supports this functionality on Linux only.
Distribution awareness (SQL nodes).
In MySQL Cluster NDB 6.3, SQL nodes can take advantage of
distribution
awareness. Here we provide a brief example showing
how to design a table to make a given class of queries
distrubtion-aware. Suppose an
NDBCLUSTER
table
t1
has the following schema:
CREATE TABLE t1 ( userid INT NOT NULL, serviceid INT NOT NULL AUTO_INCREMENT PRIMARY KEY, data VARCHAR(255) ) ENGINE=NDBCLUSTER;
Suppose further that most of the queries to be used in our
application test values of the userid
column of this table. The form of such a query looks
something like this:
SELECTcolumns
FROM t1 WHERE useridrelation
value
;
In this query, relation
represents some relational operator, such as
=
, <
,
>
, and so on. Queries using
IN
and a list of values can also be used:
SELECTcolumns
FROM t1 WHERE userid INvalue_list
;
To make use of distribution awareness, we need to make the
userid
column part of the table's
primary key, then explicitly partition the table with this
column being used as the partitioning key. (Recall that for
a partitioned table having one or more unique keys, all
columns of the table's partitioning key must also be
part of all of the unique keys—for more information
and examples, see
Section 18.5.1, “Partitioning Keys, Primary Keys, and Unique Keys”.)
In other words, the table schema should be equivalent to the
following CREATE TABLE
statement:
CREATE TABLE t1 ( userid INT NOT NULL, serviceid INT NOT NULL AUTO_INCREMENT, data VARCHAR(255), PRIMARY KEY p (userid,serviceid) ) ENGINE=NDBCLUSTER PARTITION BY KEY(userid);
When the table is partitioned in this way, all rows having
the same userid
value are found on the
same node group, and the MySQL Server can immediately select
the optimal node to use as the transaction coordinator.
Realtime extensions for multiple CPUs.
When running MySQL Cluster data nodes on hosts with
multiple processors, the realtime extensions make it
possible to give priority to the data node process and
control on which CPU cores it should operate. This can be
done using the data node configuration parameters
RealtimeScheduler
,
SchedulerExecutionTimer
,
and
SchedulerSpinTimer
.
Doing so properly can significantly lower response times
and make them much more predictable response. For more
information about using these parameters, see
Defining
Data Nodes: Realtime Performance Parameters
Fully automatic database discovery.
It is no longer a requirement for database autodiscovery
that an SQL node already be connected to the cluster at
the time that a database is created on another SQL node,
or for a CREATE DATABASE
or
CREATE
SCHEMA
statement to be issued on the new SQL
node after it joins the cluster.
Detection of NDB API client connection errors.
Beginning with MySQL Cluster NDB 6.3.20, the NDB
API's
Ndb_cluster_connection
class adds the
get_latest_error()
and
get_latest_error_msg()
methods for catching and diagnosing problems with NDB API
client connections.
Restoring specific databases, tables, or columns from a MySQL Cluster
backup.
It is now possible to exercise more fine-grained control
when restoring a MySQL Cluster from backup using
ndb_restore. Beginning with MySQL
Cluster NDB 6.3.22, you can choose to restore only
specified tables or databases, or exclude specific tables
or databases from being restored, using the new
ndb_restore options
--include-tables
,
--include-databases
,
--exclude-tables
, and
--exclude-databases
. Beginning with MySQL
Cluster NDB 6.3.26, it is also possible to restore to a
table having fewer columns than the original using the
--exclude-missing-columns
option. For
more information about all of these options, see
Section 17.4.20, “ndb_restore — Restore a MySQL Cluster Backup”.
Improved Disk Data file system configuration.
As of MySQL Cluster NDB 6.3.22, you can specify default
locations for MySQL Cluster Disk Data data files and undo
log files using the data node configuration parameters
FileSystemPathDD
,
FileSystemPathDataFiles
,
and
FileSystemPathUndoFiles
.
This eliminates the need to use symbolic links to place
Disk Data files separately from other files in data node
file systems to improve Disk Data performance. For more
information, see
Disk Data file system parameters.
Automatic creation of Disk Data log file groups and tablespaces.
Beginning with MySQL Cluster NDB 6.3.22, using the data
node configuration parameters
InitialLogFileGroup
and
InitialTablespace
,
you can cause the creation of a MySQL Cluster Disk Data
log file group, tablespace, or both, when the cluster is
first started. When using these parameters, no SQL
statements are required to create these Disk Data objects.
For more information, see
Disk
Data object creation parameters.
Configuration parameter data dumps.
Starting with MySQL Cluster NDB 6.3.25, the
ndb_config utility supports a
--configinfo
option that causes it to
dump a list of all configuration parameters supported by
the cluster, along with brief descriptions, information
about the parameters' default and permitted values,
and the sections of the config.ini
file in which the parameters apply. An additional
--xml
switch causes
ndb_config to use XML rather than
plaintext output. Using ndb_config
--configinfo
or
ndb_config --configinfo
--xml
requires no access to a running
MySQL Cluster, any other programs, or any files. For more
information and examples, see
Section 17.4.7, “ndb_config — Extract MySQL Cluster Configuration Information”.
Per-table reporting of free space on disk.
The INFORMATION_SCHEMA.FILES
table shows information about used and free space in MySQL
Cluster Disk Data data files, but this information is not
applicable to individual tables. In MySQL Cluster NDB
6.3.27 and later, the ndb_desc utility
provides two additional columns in its output that show
the amount of space allocated on disk for a given
NDB
table as well the amount
of space that remains available for additional storage of
disk-based column data for that table. For more
information, see
Section 17.4.10, “ndb_desc — Describe NDB Tables”.
Improved restart times. Optimizations in redo log handling and other file system operations introduced in MySQL Cluster NDB 6.3.28 have the potential to reduce considerably the time required for restarts. While actual performance benefits observed in production setups will naturally vary depending on database size, hardware, and other conditions, our own preliminary testing has shown that these improvements can yield startup times that are faster than those typical of previous MySQL Cluster NDB 6.3 releases by a factor of 50 or more.
Increased flexibility in online upgrade procedure. Previously, when performing an upgrade of a running MySQL cluster, the order in which the types of cluster nodes had to be upgraded was very strict. However, beginning with MySQL Cluster NDB 6.3.29, MySQL Cluster supports online upgrading of API nodes (including MySQL servers running as SQL nodes) before upgrading management nodes, data nodes, or both.
Before attempting to use this new upgrade functionality, see Section 17.5.5, “Performing a Rolling Restart of a MySQL Cluster”, for additional information, especially if you are planning an online upgrade from MySQL Cluster NDB 6.3 to MySQL Cluster NDB 7.0.
New replication conflict resolution strategy.
Beginning with MySQL Cluster NDB 6.3.31, the function
NDB$MAX_DELETE_WIN()
is available to
implement “greatest timestamp, delete wins”
conflict resolution. See
NDB$MAX_DELETE_WIN(column_name),
for more information.
New CHANGE MASTER TO option for circular replication.
Beginning with MySQL Cluster NDB 6.3.31, the
CHANGE MASTER TO
statement
supports an IGNORE_SERVER_IDS
option
which takes a comma-separated list of server IDs and
causes events originating from the corresponding servers
to be ignored. (Log rotation and log deletion events are
preserved.)
See Section 13.4.2.1, “CHANGE MASTER TO Syntax”, as well as Section 13.7.5.36, “SHOW SLAVE STATUS Syntax”, for more information.
Heartbeat thread policy and priority.
Beginning with MySQL Cluster NDB 6.3.32, a new
configuration parameter
HeartbeatThreadPriority
makes it
possible to set the policy and the priority for the
heartbeat thread on management and API nodes.
Improved access to partitioning information.
The ndb_desc utility now provides
additional information about the partitioning of data
stored in MySQL Cluster. Beginning with MySQL Cluster NDB
6.3.32, the --blob-info
option causes this program to include partition
information for BLOB
tables
in its output. Beginning with MySQL Cluster NDB 6.3.33,
the --extra-node-info
option causes ndb_desc to include
information about data distribution (that is, which table
fragments are stored on which data nodes). Each of these
options also requires the use of the
--extra-partition-info
option.
Information about partition-to-node mappings can also be
obtained using the
Table::getFragmentNodes()
method, also added in MySQL Cluster NDB 6.3.33.
Replication attribute promotion and demotion.
Beginning with MySQL Cluster NDB 6.3.33, MySQL Cluster
Replication supports attribute promotion and demotion when
replicating between columns of different but similar types
on the master and the slave. For example, it is possible
to promote an INT
column on
the master to a BIGINT
column on the slave, and to demote a
TEXT
column to a
VARCHAR
column.
The implementation of type demotion distinguishes between
lossy and non-lossy type conversions, and their use on the
slave can be controlled by setting the
slave_type_conversions
global server system variable.
For more information, see Attribute promotion and demotion (MySQL Cluster).
Incompatible change in NDB API event reporting.
Beginning with MySQL Cluster NDB 6.3.34, DDL events are no
longer reported on Event
objects by default. Instead such event reporting must be
enabled explicitly using the
Event::setReport()
method. For more information, see
Event::setReport(), and
The Event::EventReport Type.
Heartbeat ordering.
Beginning with MySQL Cluster NDB 6.3.35, it is possible to
set a specific order for transmission of heartbeats
between data nodes, using the
HeartbeatOrder
data node configuration parameter introduced in this
version. This parameter can be useful in situations where
multiple data nodes are running on the same host and a
temporary disruption in connectivity between hosts would
otherwise cause the loss of a node group (and thus failure
of the cluster).
Relaxed ndb_restore column comparison rules. When restoring data, ndb_restore compares the attributes of a column for equality with the definition of the column in the target table. However, not all of these attributes need to be the same for ndb_restore to be meaningful, safe and useful. Beginning with MySQL Cluster NDB 6.3.35, ndb_restore automatically ignores differences in certain column attributes which do not necessarily have to match between the version of the column in a backup and the version of that column in the MySQL Cluster to which the column data is being restored. These attributes include the following:
COLUMN_FORMAT
setting
(FIXED
, DYNAMIC
,
or DEFAULT
)
STORAGE
setting
(MEMORY
or DISK
)
The default value
The distribution key
In such cases, ndb_restore reports any such differences to minimize any chance of user error.
--add-drop-trigger option for mysqldump.
Beginning with MySQL Cluster NDB 6.3.38, this option can
be used to force all CREATE
TRIGGER
statements in
mysqldump output to be preceded by a
DROP TRIGGER
IF EXISTS
statement.
Skipping corrupted tables in NDB native backups.
Beginning with MySQL Cluster NDB 6.3.40, you can cause
ndb_restore to ignore tables that are
corrupted due to missing blob parts tables by using the
--skip-broken-objects
option. When this option is used, such tables are skipped,
and the restoration of any remaining uncorrupted tables in
the backup continues.
Restoring from a NDB native backup to a differently-named database.
MySQL Cluster NDB 6.3.41 adds a new
--rewrite-database
option to ndb_restore, which makes it
possible to restore to a database having a different name
from that of the database in the backup. The option can be
used multiple times, and it is possible to restore from
more than one source database in the backup to a single
target database (although no protection against table or
other object name collision is provided).
See Section 17.4.20, “ndb_restore — Restore a MySQL Cluster Backup”, for more information.
The following list provides an overview of significant feature additions and changes made in MySQL Cluster NDB 6.2. All of the changes in this list are also available in MySQL Cluster NDB 6.3 . For more detailed information about all feature changes and bugfixes made in MySQL Cluster NDB 6.2, see http://dev.mysql.com/doc/relnotes/mysql-cluster/7.1/en/mysql-cluster-news-6-2.html.
Enhanced backup status reporting.
Backup status reporting has been improved, aided in part
by the introduction of a
BackupReportFrequency
configuration parameter.
Multiple cluster connections per SQL node.
A single MySQL server acting as a MySQL Cluster SQL node
can employ multiple connections to the cluster using the
--ndb-cluster-connection-pool
startup
option for mysqld. This option is
described in
MySQL
Cluster-Related Command Options for
mysqld:
--ndb-cluster-connection-pool
option.
New data access interface.
The NdbRecord
interface
provides a new and simplified data handler for use in NDB
API applications.
New reporting commands.
The new management client REPORT
BackupStatus
and REPORT
MemoryUsage
commands provide better access to
information about the status of MySQL Cluster backups and
how much memory is being used by MySQL Cluster for data
and index storage. See
Section 17.5.2, “Commands in the MySQL Cluster Management Client”, for
more information about the REPORT
commands. In addition, in-progress status reporting is
provided by the ndb_restore utility;
see Section 17.4.20, “ndb_restore — Restore a MySQL Cluster Backup”.
Improved memory allocation and configuration.
Memory is now allocated by the
NDB
kernel to tables on a
page-by-page basis, which significantly reduces the memory
overhead required for maintaining
NDBCLUSTER
tables. In
addition, the
MaxAllocate
configuration parameter now makes it possible to set the
maximum size of the allocation unit used for table memory.
Choice of fixed-width or variable-width columns.
You can control whether fixed-width or variable-width
storage is used for a given column of an
NDB
table by employing of the
COLUMN_FORMAT
specifier as part of the
column's definition in a CREATE
TABLE
or ALTER
TABLE
statement. In addition, the ability to
control whether a given column of an
NDB
table is stored in memory
or on disk, using the STORAGE
specifier
as part of the column's definition in a
CREATE TABLE
or
ALTER TABLE
statement. For
more information, see Section 13.1.17, “CREATE TABLE Syntax”, and
Section 13.1.7, “ALTER TABLE Syntax”.
Controlling management client connections.
The --bind-address
cluster management
server startup option makes it possible to restrict
management client connections to
ndb_mgmd to a single host (IP address
or host name) and port, which can make MySQL Cluster
management operations more secure. For more information
about this option, see
Section 17.4.4, “ndb_mgmd — The MySQL Cluster Management Server Daemon”.
Micro-GCPs.
Due to a change in the protocol for handling of global
checkpoints (GCPs handled in this manner sometimes being
referred to as “micro-GCPs”), it is now
possible to control how often the GCI number is updated,
and how often global checkpoints are written to disk,
using the
TimeBetweenEpochs
and
TimeBetweenEpochsTimeout
configuration parameters. This improves the reliability
and performance of MySQL Cluster Replication.
Core online schema change support.
Support for the online ALTER
TABLE
operations ADD COLUMN
,
ADD INDEX
, and DROP
INDEX
is available. When the
ONLINE
keyword is used, the
ALTER TABLE
is noncopying,
which means that indexes do not have to be re-created,
which provides these benefits:
Single user mode is no longer required for
ALTER TABLE
operations
that can be performed online.
Transactions can continue during
ALTER TABLE
operations
that can be performed online.
Tables being altered online are not locked against access by other SQL nodes.
However, such tables are locked against other operations
on the same SQL node for the
duration of the ALTER
TABLE
. We are working to overcome this
limitation in a future MySQL Cluster release.
Online CREATE INDEX
and
DROP INDEX
statements are
also supported. Online changes can be suppressed using the
OFFLINE
key word. See
Section 13.1.7.2, “ALTER TABLE Online Operations in MySQL Cluster”,
Section 13.1.13, “CREATE INDEX Syntax”, and
Section 13.1.24, “DROP INDEX Syntax”, for more detailed information.
mysql.ndb_binlog_index improvements.
More information has been added to the
mysql.ndb_binlog_index
table so that it
is possible to determine which originating epochs have
been applied inside an epoch. This is particularly useful
for 3-way replication. See
Section 17.6.4, “MySQL Cluster Replication Schema and Tables”, for
more information.
Epoch lag control.
The
MaxBufferedEpochs
data node configuration parameter provides a means to
control the maximum number of unprocessed epochs by which
a subscribing node can lag. Subscribers which exceed this
number are disconnected and forced to reconnect.
Fully automatic database discovery.
It is no longer a requirement for database autodiscovery
that an SQL node already be connected to the cluster at
the time that a database is created on another SQL node,
or for a CREATE DATABASE
or
CREATE
SCHEMA
statement to be issued on the new SQL
node after it joins the cluster.
Multiple data node processes per host. In earlier MySQL Cluster release series, we did not support MySQL Cluster deployments in production where more than one ndbd process was run on a single physical machine. However, beginning with MySQL Cluster NDB 6.2.0, you can use multiple data node processes on a single host.
A multi-threaded version of ndbd tailored for use on hosts with multiple CPUs or cores was introduced in MySQL Cluster NDB 7.0. See Section 17.1.4.3, “MySQL Cluster Development in MySQL Cluster NDB 7.0”, and Section 17.4.3, “ndbmtd — The MySQL Cluster Data Node Daemon (Multi-Threaded)”, for more information.
Improved Disk Data file system configuration.
As of MySQL Cluster NDB 6.2.17, you can specify default
locations for MySQL Cluster Disk Data data files and undo
log files using the data node configuration parameters
FileSystemPathDD
,
FileSystemPathDataFiles
,
and
FileSystemPathUndoFiles
.
This eliminates the need to use symbolic links to place
Disk Data files separately from other files in data node
file systems to improve Disk Data performance. For more
information, see
Disk Data file system parameters.
Automatic creation of Disk Data log file groups and tablespaces.
Beginning with MySQL Cluster NDB 6.2.17, using the data
node configuration parameters
InitialLogFileGroup
and
InitialTablespace
,
you can cause the creation of a MySQL Cluster Disk Data
log file group, tablespace, or both, when the cluster is
first started. When using these parameters, no SQL
statements are required to create these Disk Data objects.
Improved access to partitioning information.
The ndb_desc utility now provides
additional information about the partitioning of data
stored in MySQL Cluster. Beginning with MySQL Cluster NDB
6.2.19, the
--extra-node-info
option
causes ndb_desc to include information
about data distribution (that is, which table fragments
are stored on which data nodes). This option also requires
the use of the
--extra-partition-info
option.
Information about partition-to-node mappings can also be
obtained using the
Table::getFragmentNodes()
method, also added in MySQL Cluster NDB 6.2.19.
New CHANGE MASTER TO option for circular replication.
Beginning with MySQL Cluster NDB 6.2.19, the
CHANGE MASTER TO
statement
supports an IGNORE_SERVER_IDS
option
which takes a comma-separated list of server IDs and
causes events originating from the corresponding servers
to be ignored. (Log rotation and log deletion events are
preserved.)
See Section 13.4.2.1, “CHANGE MASTER TO Syntax”, as well as Section 13.7.5.36, “SHOW SLAVE STATUS Syntax”, for more information.
The following list provides an overview of significant feature additions and changes made in MySQL Cluster NDB 6.1. All of the changes in this list are also available in MySQL Cluster NDB 6.2 and 6.3 releases. For detailed information about all changes made in MySQL Cluster NDB 6.1, see http://dev.mysql.com/doc/relnotes/mysql-cluster/7.1/en/mysql-cluster-news-6-1.html.
Increased number of cluster nodes. The maximum number of all nodes in a MySQL Cluster has been increased to 255. For more information, see Section 17.1.6.10, “Limitations Relating to Multiple MySQL Cluster Nodes”.
Disabling arbitration.
It is now possible to disable arbitration by setting
ArbitrationRank=0
on all cluster
management and SQL nodes. For more information, see
Defining
the Management Server:
ArbitrationRank
and
Defining
SQL and Other API Nodes:
ArbitrationRank
.
Additional DUMP commands.
New management client DUMP
commands
provide help with tracking transactions, scan operations,
and locks. See
Section 17.5.2, “Commands in the MySQL Cluster Management Client”, and
MySQL Cluster Management Client DUMP Commands, for more
information.
Faster Disk Data backups. Improvements in backups of Disk Data tables can yield a 10 to 15% increase in backup speed of Disk Data tables.
Batched slave updates.
Batching of updates on cluster replication slaves, enabled
using the --slave-allow-batching
option
for mysqld, increases replication
efficiency. For more information, see
Section 17.6.6, “Starting MySQL Cluster Replication (Single Replication Channel)”.
A number of features for MySQL Cluster were implemented in MySQL 5.1 through MySQL 5.1.23, when support for MySQL Cluster was moved to MySQL Cluster NDB. All of the features in the following list are also available in all MySQL Cluster NDB (6.1 and later) releases.
Integration of MySQL Cluster into MySQL Replication.
MySQL Cluster Replication makes it possible to replicate
from one MySQL Cluster to another. Updates on any SQL node
(MySQL server) in the cluster acting as the master are
replicated to the slave cluster; the state of the slave
side remains consistent with the cluster acting as the
master. This is sometimes referred to as
asynchronous replication between
clusters, providing geographic
redundancy. It is also possible to replicate
from a MySQL Cluster acting as the master to a standalone
MySQL server acting as the slave, or from a standalone
MySQL master server to a slave cluster; in either of these
cases, the standalone MySQL server uses a storage engine
other than NDBCLUSTER
.
Multi-master replication setups such as circular
replication are also supported.
Support for storage of rows on disk.
Storage of NDBCLUSTER
table
data on disk is now supported. Indexed columns, including
the primary key hash index, must still be stored in RAM;
however, all other columns can be stored on disk.
Variable-size columns.
In MySQL 5.0, an NDBCLUSTER
table column defined as VARCHAR(255)
used 260 bytes of storage independent of what was stored
in any particular record. In MySQL 5.1 Cluster tables,
only the portion of the column actually taken up by the
record is stored. This makes possible a significant
reduction in space requirements for such columns as
compared to previous release series—by a factor of
up to 5 in many cases.
User-defined partitioning.
Users can define partitions based on columns that are part
of the primary key. It is possible to partition
NDB
tables based on
KEY
and LINEAR KEY
schemes. This feature is also available for many other
MySQL storage engines, which support additional
partitioning types that are not available with
NDBCLUSTER
tables.
For additional general information about user-defined partitioning in MySQL 5.1, see Chapter 18, Partitioning. Specifics of partitioning types are discussed in Section 18.2, “Partitioning Types”.
The MySQL Server can also determine whether it is possible
to “prune away” some of the partitions from the
WHERE
clause, which can greatly speed up
some queries. See Section 18.4, “Partition Pruning”,
for information about designing tables and queries to take
advantage of partition pruning.
Autodiscovery of table schema changes.
In MySQL 5.0, it was necessary to issue a
FLUSH
TABLES
statement or a “dummy”
SELECT
for new
NDBCLUSTER
tables or changes
made to schemas of existing
NDBCLUSTER
tables on one SQL
node to be visible on the cluster's other SQL nodes.
In MySQL 5.1, this is no longer necessary; new Cluster
tables and changes in the definitions of existing
NDBCLUSTER
tables made on one
SQL node are immediately visible to all SQL nodes
connected to the cluster.
When creating a new database, it is still necessary in
MySQL 5.1 to issue a CREATE
DATABASE
or
CREATE
SCHEMA
statement on each SQL node in the
cluster.
Distribution awareness (NDB API). Distribution awareness is a mechanism by which the best data node is automatically selected to be queried for information. (Conceptually, it is similar in some ways to partition pruning (see Section 18.4, “Partition Pruning”). To take advantage of distribution awareness, you should do the following:
Determine which table column is most likely to be used for finding matching records.
Make this column part of the table's primary key.
Explicitly partition the table by
KEY
, using this column as the
table' partitioning key.
Following these steps causes records with the same value for the partitioning column to be stored on the same partition (that is, in the same node group). When reading data, transactions are begun on the data node actually having the desired rows instead of this node being determined by the usual round-robin mechanism.
To see a measureable impact on performance, the cluster must have at least four data nodes, since, with only two data nodes, both data nodes have exactly the same data.
Using distribution awareness can yield performance increase of as great as 45% when using four data nodes, and possibly more when using a greater number of data nodes.
In mainline MySQL 5.1 releases, distribution awareness was supported only when using the NDB API; support was added for SQL and API nodes in MySQL Cluster NDB 6.3 (see Section 17.1.4.4, “MySQL Cluster Development in MySQL Cluster NDB 6.3”, which includes an example showing how to create a table to take advantage of distribution awareness).
Online adding and dropping of indexes.
For ALTER TABLE
statements,
ADD INDEX
and DROP
INDEX
operations for dynamic (variable-width)
columns on NDB
tables are now
performed as online operations (no table copying). This is
also true for CREATE INDEX
and DROP INDEX
. As a
result, these operations are now performed much more
quickly than previously.
For more information, see Section 13.1.7.2, “ALTER TABLE Online Operations in MySQL Cluster”, Section 13.1.13, “CREATE INDEX Syntax”, and Section 13.1.24, “DROP INDEX Syntax”.
See Section 17.1.6.11, “Previous MySQL Cluster Issues Resolved in MySQL 5.1, MySQL Cluster NDB 6.x, and MySQL Cluster NDB 7.x”, for more information.
MySQL Server offers a number of choices in storage engines. Since
both NDBCLUSTER
and
InnoDB
can serve as transactional
MySQL storage engines, users of MySQL Server sometimes become
interested in MySQL Cluster. They see
NDB
as a possible alternative or
upgrade to the default InnoDB
storage
engine in MySQL 5.5. While NDB
and
InnoDB
share common characteristics,
there are differences in architecture and implementation, so that
some existing MySQL Server applications and usage scenarios can be
a good fit for MySQL Cluster, but not all of them.
In this section, we discuss and compare some characteristics of
the NDB
storage engine used by MySQL
Cluster with InnoDB
used in MySQL 5.1
and MySQL 5.5. The next few sections provide a technical
comparison. In many instances, decisions about when and where to
use MySQL Cluster must be made on a case-by-case basis, taking all
factors into consideration. While it is beyond the scope of this
documentation to provide specifics for every conceivable usage
scenario, we also attempt to offer some very general guidance on
the relative suitability of some common types of applications for
NDB
as opposed to
InnoDB
backends.
While it is possible to use InnoDB
tables with MySQL Cluster, such tables are not clustered. MySQL
Cluster NDB 7.1 releases use a mysqld based on
MySQL 5.1 which includes InnoDB
1.0; MySQL
Cluster NDB 7.2.1 and later are based on MySQL Server 5.5, which
includes support for InnoDB
1.1. It is also not
possible to use programs or libraries from a MySQL Cluster NDB 6.x
or MySQL Cluster NDB 7.x distribution with MySQL Server 5.1 or
MySQL Server 5.5, or the reverse.
While it is also true that some types of common business
applications can be run either on MySQL Cluster or on MySQL Server
(most likely using the InnoDB
storage
engine), there are some important architectural and implementation
differences. Section 17.1.5.1, “Differences Between the NDB and InnoDB Storage Engines”,
provides a summary of the these differences. Due to the
differences, some usage scenarios are clearly more suitable for
one engine or the other; see
Section 17.1.5.2, “NDB and InnoDB Workloads”. This in turn
has an impact on the types of applications that better suited for
use with NDB
or
InnoDB
. See
Section 17.1.5.3, “NDB and InnoDB Feature Usage Summary”, for a comparison
of the relative suitability of each for use in common types of
database applications.
For information about the relative characteristics of the
NDB
and
MEMORY
storage engines, see
When to Use MEMORY or MySQL Cluster.
See Chapter 14, Storage Engines, for additional information about MySQL storage engines.
The MySQL Cluster NDB
storage
engine is implemented using a distributed, shared-nothing
architecture, which causes it to behave differently from
InnoDB
in a number of ways. For
those unaccustomed to working with
NDB
, unexpected behaviors can arise
due to its distributed nature with regard to transactions,
foreign keys, joins, and other characteristics. These are shown
in the following table:
Feature | MySQL Cluster | |
---|---|---|
MySQL Server Version |
| MySQL Cluster NDB 6.3, 7.0, 7.1: 5.1 MySQL Cluster NDB 7.2: 5.5 MySQL Cluster NDB 7.3: 5.6 |
|
|
NDB 6.3, 7.0:
NDB 7.1:
NDB 7.2:
|
Storage Limits | 64TB | 3TB (Practical upper limit based on 48 data nodes with 64GB RAM each; can be increased with disk-based data and BLOBs) |
Foreign Keys | Yes |
Prior to MySQL Cluster NDB 7.3: No. (Ignored, as with
Available in MySQL Cluster NDB 7.3 and later. |
Transactions | All standard types | |
MVCC | Yes | No |
Data Compression | Yes | No (MySQL Cluster checkpoint and backup files can be compressed) |
Large Row Support (> 14K) |
Supported for (Using these types to store very large amounts of data can lower MySQL Cluster performance) | |
Replication Support | Asynchronous and semisynchronous replication using MySQL Replication | Automatic synchronous replication within a MySQL Cluster. Asynchronous replication between MySQL Clusters, using MySQL Replication |
Scaleout for Read Operations | Yes (MySQL Replication) | Yes (Automatic partitioning in MySQL Cluster; MySQL Replication) |
Scaleout for Write Operations | Requires application-level partitioning (sharding) | Yes (Automatic partitioning in MySQL Cluster is transparent to applications) |
High Availability (HA) | Requires additional software | Yes (Designed for 99.999% uptime) |
Node Failure Recovery and Failover | Requires additional software | Automatic (Key element in MySQL Cluster architecture) |
Time for Node Failure Recovery | 30 seconds or longer | Typically < 1 second |
Real-Time Performance | No | Yes |
In-Memory Tables | No | Yes (Some data can optionally be stored on disk; both in-memory and disk data storage are durable) |
NoSQL Access to Storage Engine | Native memcached interface in development (see the MySQL Dev Zone article MySQL Cluster 7.2 (DMR2): NoSQL, Key/Value, Memcached) | Yes Multiple APIs, including C++, HTTP/REST, Java and JPA (MySQL Cluster NDB 7.1 and later); Memcached (MySQL Cluster NDB 7.2 and later); Node.js under development (MySQL Cluster NDB 7.3) |
Concurrent and Parallel Writes | Not supported | Up to 48 writers, optimized for concurrent writes |
Conflict Detection and Resolution (Multiple Replication Masters) | No | Yes |
Hash Indexes | No | Yes |
Online Addition of Nodes | Read-only replicas using MySQL Replication | Yes (all node types) |
Online Upgrades | No | Yes |
Online Schema Modifications | No. | Yes. |
MySQL Cluster has a range of unique attributes that make it
ideal to serve applications requiring high availability, fast
failover, high throughput, and low latency. Due to its
distributed architecture and multi-node implementation, MySQL
Cluster also has specific constraints that may keep some
workloads from performing well. A number of major differences in
behavior between the NDB
and
InnoDB
storage engines with regard
to some common types of database-driven application workloads
are shown in the following table::
Workload |
MySQL Cluster ( | |
---|---|---|
High-Volume OLTP Applications | Yes | Yes |
DSS Applications (data marts, analytics) | Yes | Limited (Join operations across OLTP datasets not exceeding 3TB in size) |
Custom Applications | Yes | Yes |
Packaged Applications | Yes | Limited (should be mostly primary key access).
Note
MySQL Cluster NDB 7.3 supports foreign keys. |
In-Network Telecoms Applications (HLR, HSS, SDP) | No | Yes |
Session Management and Caching | Yes | Yes |
E-Commerce Applications | Yes | Yes |
User Profile Management, AAA Protocol | Yes | Yes |
When comparing application feature requirements to the
capabilities of InnoDB
with
NDB
, some are clearly more
compatible with one storage engine than the other.
The following table lists supported application features according to the storage engine to which each feature is typically better suited.
Preferred application requirements for
|
Preferred application requirements for
|
---|---|
|
|
In the sections that follow, we discuss known limitations in
current releases of MySQL Cluster as compared with the features
available when using the MyISAM
and
InnoDB
storage engines. If you check the
“Cluster” category in the MySQL bugs database at
http://bugs.mysql.com, you can find known bugs in
the following categories under “MySQL Server:” in the
MySQL bugs database at http://bugs.mysql.com, which
we intend to correct in upcoming releases of MySQL Cluster:
MySQL Cluster
Cluster Direct API (NDBAPI)
Cluster Disk Data
Cluster Replication
ClusterJ
This information is intended to be complete with respect to the conditions just set forth. You can report any discrepancies that you encounter to the MySQL bugs database using the instructions given in Section 1.7, “How to Report Bugs or Problems”. If we do not plan to fix the problem in MySQL Cluster NDB 7.1, we will add it to the list.
See Section 17.1.6.11, “Previous MySQL Cluster Issues Resolved in MySQL 5.1, MySQL Cluster NDB 6.x, and MySQL Cluster NDB 7.x” for a list of issues in MySQL Cluster in MySQL 5.0 that have been resolved in the current version.
Limitations and other issues specific to MySQL Cluster Replication are described in Section 17.6.3, “Known Issues in MySQL Cluster Replication”.
Some SQL statements relating to certain MySQL features produce
errors when used with NDB
tables,
as described in the following list:
Temporary tables.
Temporary tables are not supported. Trying either to
create a temporary table that uses the
NDB
storage engine or to
alter an existing temporary table to use
NDB
fails with the error
Table storage engine 'ndbcluster' does not
support the create option 'TEMPORARY'.
Indexes and keys in NDB tables. Keys and indexes on MySQL Cluster tables are subject to the following limitations:
Column width.
Attempting to create an index on an
NDB
table column whose width is
greater than 3072 bytes succeeds, but only the first
3072 bytes are actually used for the index. In such
cases, a warning Specified key was too
long; max key length is 3072 bytes is
issued, and a SHOW CREATE
TABLE
statement shows the length of the
index as 3072.
TEXT and BLOB columns.
You cannot create indexes on
NDB
table columns that
use any of the TEXT
or
BLOB
data types.
FULLTEXT indexes.
The NDB
storage engine
does not support FULLTEXT
indexes,
which are possible for MyISAM
tables only.
However, you can create indexes on
VARCHAR
columns of
NDB
tables.
USING HASH keys and NULL.
Using nullable columns in unique keys and primary keys
means that queries using these columns are handled as
full table scans. To work around this issue, make the
column NOT NULL
, or re-create the
index without the USING HASH
option.
Prefixes.
There are no prefix indexes; only entire columns can
be indexed. (The size of an NDB
column index is always the same as the width of the
column in bytes, up to and including 3072 bytes, as
described earlier in this section. Also see
Section 17.1.6.6, “Unsupported or Missing Features in MySQL Cluster”,
for additional information.)
BIT columns.
A BIT
column cannot be
a primary key, unique key, or index, nor can it be
part of a composite primary key, unique key, or index.
AUTO_INCREMENT columns.
Like other MySQL storage engines, the
NDB
storage engine can
handle a maximum of one
AUTO_INCREMENT
column per table.
However, in the case of a Cluster table with no
explicit primary key, an
AUTO_INCREMENT
column is
automatically defined and used as a
“hidden” primary key. For this reason,
you cannot define a table that has an explicit
AUTO_INCREMENT
column unless that
column is also declared using the PRIMARY
KEY
option. Attempting to create a table
with an AUTO_INCREMENT
column that
is not the table's primary key, and using the
NDB
storage engine, fails
with an error.
MySQL Cluster and geometry data types.
Geometry data types (WKT
and
WKB
) are supported in
NDB
tables in MySQL 5.1
(including MySQL Cluster NDB 6.X and 7.X through 7.1).
However, spatial indexes are not supported.
Character sets and binary log files.
Currently, the ndb_apply_status
and
ndb_binlog_index
tables are created
using the latin1
(ASCII) character set.
Because names of binary logs are recorded in this table,
binary log files named using non-Latin characters are not
referenced correctly in these tables. This is a known
issue, which we are working to fix. (Bug #50226)
To work around this problem, use only Latin-1 characters
when naming binary log files or setting any the
--basedir
,
--log-bin
, or
--log-bin-index
options.
Creating NDBCLUSTER tables with user-defined partitioning.
Support for user-defined partitioning for MySQL Cluster is
restricted to [LINEAR
]
KEY
partitioning. Beginning with MySQL
5.1.12, using any other partitioning type with
ENGINE=NDB
or
ENGINE=NDBCLUSTER
in a
CREATE TABLE
statement
results in an error.
Default partitioning scheme.
As of MySQL 5.1.6, all MySQL Cluster tables are by default
partitioned by KEY
using the table's
primary key as the partitioning key. If no primary key is
explicitly set for the table, the “hidden”
primary key automatically created by the
NDBCLUSTER
storage engine is
used instead. For additional discussion of these and
related issues, see Section 18.2.4, “KEY Partitioning”.
Beginning with MySQL Cluster NDB 6.2.18, MySQL Cluster NDB
6.3.25, and MySQL Cluster NDB 7.0.6,
CREATE TABLE
and
ALTER TABLE
statements that
would cause a user-partitioned
NDBCLUSTER
table not to meet
either or both of the following two requirements are not
permitted, and fail with an error (Bug #40709):
The table must have an explicit primary key.
All columns listed in the table's partitioning expression must be part of the primary key.
Exception.
If a user-partitioned
NDBCLUSTER
table is created
using an empty column-list (that is, using
PARTITION BY [LINEAR] KEY()
), then no
explicit primary key is required.
Maximum number of partitions for NDBCLUSTER tables.
The maximum number of partitions that can defined for a
NDBCLUSTER
table when
employing user-defined partitioning is 8 per node group.
(See Section 17.1.2, “MySQL Cluster Nodes, Node Groups, Replicas, and Partitions”, for
more information about MySQL Cluster node groups.
DROP PARTITION not supported.
It is not possible to drop partitions from
NDB
tables using
ALTER TABLE ... DROP PARTITION
. The
other partitioning extensions to
ALTER
TABLE
—ADD PARTITION
,
REORGANIZE PARTITION
, and
COALESCE PARTITION
—are supported
for Cluster tables, but use copying and so are not
optimized. See
Section 18.3.1, “Management of RANGE and LIST Partitions” and
Section 13.1.7, “ALTER TABLE Syntax”.
Row-based replication.
When using row-based replication with MySQL Cluster,
binary logging cannot be disabled. That is, the
NDB
storage engine ignores
the value of sql_log_bin
.
(Bug #16680)
In this section, we list limits found in MySQL Cluster that either differ from limits found in, or that are not found in, standard MySQL.
Memory usage and recovery.
Memory consumed when data is inserted into an
NDB
table is not automatically
recovered when deleted, as it is with other storage engines.
Instead, the following rules hold true:
A DELETE
statement on an
NDB
table makes the memory
formerly used by the deleted rows available for re-use by
inserts on the same table only. However, this memory can be
made available for general re-use by performing a rolling
restart of the cluster. See
Section 17.5.5, “Performing a Rolling Restart of a MySQL Cluster”.
Beginning with MySQL Cluster NDB 6.3.7, this limitation can
be overcome using OPTIMIZE
TABLE
. See
Section 17.1.6.11, “Previous MySQL Cluster Issues Resolved in MySQL 5.1, MySQL Cluster NDB
6.x, and MySQL Cluster NDB 7.x”, for
more information.
A DROP TABLE
or
TRUNCATE TABLE
operation on
an NDB
table frees the memory
that was used by this table for re-use by any
NDB
table, either by the same
table or by another NDB
table.
Recall that TRUNCATE TABLE
drops and re-creates the table. See
Section 13.1.34, “TRUNCATE TABLE Syntax”.
Limits imposed by the cluster's configuration. A number of hard limits exist which are configurable, but available main memory in the cluster sets limits. See the complete list of configuration parameters in Section 17.3.2, “MySQL Cluster Configuration Files”. Most configuration parameters can be upgraded online. These hard limits include:
Database memory size and index memory size
(DataMemory
and
IndexMemory
,
respectively).
DataMemory
is
allocated as 32KB pages. As each
DataMemory
page
is used, it is assigned to a specific table; once
allocated, this memory cannot be freed except by
dropping the table.
See Section 17.3.2.6, “Defining MySQL Cluster Data Nodes”, for more information.
The maximum number of operations that can be performed
per transaction is set using the configuration
parameters
MaxNoOfConcurrentOperations
and
MaxNoOfLocalOperations
.
Bulk loading, TRUNCATE
TABLE
, and ALTER
TABLE
are handled as special cases by
running multiple transactions, and so are not subject
to this limitation.
Different limits related to tables and indexes. For
example, the maximum number of ordered indexes in the
cluster is determined by
MaxNoOfOrderedIndexes
,
and the maximum number of ordered indexes per table is
16.
Node and data object maximums. The following limits apply to numbers of cluster nodes and metadata objects:
The maximum number of data nodes is 48.
A data node must have a node ID in the range of 1 to 48, inclusive. (Previous to MySQL Cluster NDB 6.1.1, management and API nodes were restricted to the range 1 to 63 inclusive as a node ID; starting with MySQL Cluster NDB 6.1.1, management and API nodes may use node IDs in the range 1 to 255, inclusive.)
Prior to MySQL Cluster NDB 6.1.1, the total maximum number of nodes in a MySQL Cluster was 63. Beginning with MySQL Cluster NDB 6.1.1, the total maximum number of nodes in a MySQL Cluster is 255. In either case, this number includes all SQL nodes (MySQL Servers), API nodes (applications accessing the cluster other than MySQL servers), data nodes, and management servers.
The maximum number of metadata objects in current versions of MySQL Cluster is 20320. This limit is hard-coded.
See Section 17.1.6.11, “Previous MySQL Cluster Issues Resolved in MySQL 5.1, MySQL Cluster NDB 6.x, and MySQL Cluster NDB 7.x”, for more information.
A number of limitations exist in MySQL Cluster with regard to the handling of transactions. These include the following:
Transaction isolation level.
The NDBCLUSTER
storage engine
supports only the READ
COMMITTED
transaction isolation level.
(InnoDB
, for example, supports
READ COMMITTED
,
READ UNCOMMITTED
,
REPEATABLE READ
, and
SERIALIZABLE
.) See
Section 17.5.3.4, “MySQL Cluster Backup Troubleshooting”,
for information on how this can affect backing up and
restoring Cluster databases.)
Transactions and BLOB or TEXT columns.
NDBCLUSTER
stores only part
of a column value that uses any of MySQL's
BLOB
or
TEXT
data types in the
table visible to MySQL; the remainder of the
BLOB
or
TEXT
is stored in a
separate internal table that is not accessible to MySQL.
This gives rise to two related issues of which you should
be aware whenever executing
SELECT
statements on tables
that contain columns of these types:
For any SELECT
from a
MySQL Cluster table: If the
SELECT
includes a
BLOB
or
TEXT
column, the
READ COMMITTED
transaction isolation level is converted to a read with
read lock. This is done to guarantee consistency.
Prior to MySQL Cluster NDB 7.0.12, for any
SELECT
which used a
primary key lookup or unique key lookup to retrieve any
columns that used any of the
BLOB
or
TEXT
data types and that
was executed within a transaction, a shared read lock
was held on the table for the duration of the
transaction—that is, until the transaction was
either committed or aborted.
In MySQL Cluster NDB 7.0.12 and later, for primary key
lookups, the lock is released as soon as all
BLOB
or
TEXT
data has been read.
(Bug #49190) However, for unique key lookups, the shared
lock continues to be held for the lifetime of the
transaction.
This issue does not occur for queries that use index or
table scans, even against
NDB
tables having
BLOB
or
TEXT
columns.
For example, consider the table t
defined by the following CREATE
TABLE
statement:
CREATE TABLE t ( a INT NOT NULL AUTO_INCREMENT PRIMARY KEY, b INT NOT NULL, c INT NOT NULL, d TEXT, INDEX i(b), UNIQUE KEY u(c) ) ENGINE = NDB,
Either of the following queries on t
causes a shared read lock, because the first query uses
a primary key lookup and the second uses a unique key
lookup:
SELECT * FROM t WHERE a = 1; SELECT * FROM t WHERE c = 1;
However, none of the four queries shown here causes a shared read lock:
SELECT * FROM t WHERE b 1; SELECT * FROM t WHERE d = '1'; SELECT * FROM t; SELECT b,c WHERE a = 1;
This is because, of these four queries, the first uses
an index scan, the second and third use table scans, and
the fourth, while using a primary key lookup, does not
retrieve the value of any
BLOB
or
TEXT
columns.
You can help minimize issues with shared read locks by
avoiding queries that use unique key lookups (or primary
key lookups in MySQL Cluster NDB 7.0.11 and earlier)
that retrieve BLOB
or
TEXT
columns, or, in
cases where such queries are not avoidable, by
committing transactions as soon as possible afterward.
Rollbacks. There are no partial transactions, and no partial rollbacks of transactions. A duplicate key or similar error causes the entire transaction to be rolled back.
This behavior differs from that of other transactional
storage engines such as InnoDB
that may roll back individual statements.
Transactions and memory usage. As noted elsewhere in this chapter, MySQL Cluster does not handle large transactions well; it is better to perform a number of small transactions with a few operations each than to attempt a single large transaction containing a great many operations. Among other considerations, large transactions require very large amounts of memory. Because of this, the transactional behavior of a number of MySQL statements is effected as described in the following list:
TRUNCATE TABLE
is not
transactional when used on
NDB
tables. If a
TRUNCATE TABLE
fails to
empty the table, then it must be re-run until it is
successful.
DELETE FROM
(even with no
WHERE
clause) is
transactional. For tables containing a great many rows,
you may find that performance is improved by using
several DELETE FROM ... LIMIT ...
statements to “chunk” the delete operation.
If your objective is to empty the table, then you may
wish to use TRUNCATE
TABLE
instead.
LOAD DATA statements.
LOAD DATA
INFILE
is not transactional when used on
NDB
tables.
When executing a
LOAD DATA
INFILE
statement, the
NDB
engine performs
commits at irregular intervals that enable better
utilization of the communication network. It is not
possible to know ahead of time when such commits take
place.
LOAD DATA FROM MASTER
is
not supported in MySQL Cluster.
ALTER TABLE and transactions.
When copying an NDB
table
as part of an ALTER
TABLE
, the creation of the copy is
nontransactional. (In any case, this operation is
rolled back when the copy is deleted.)
Transactions and the COUNT() function.
When using MySQL Cluster Replication, it is not possible
to guarantee the transactional consistency of the
COUNT()
function on the slave. In other
words, when performing on the master a series of
statements (INSERT
,
DELETE
, or both) that
changes the number of rows in a table within a single
transaction, executing SELECT COUNT(*) FROM
queries on the
slave may yield intermediate results. This is due to the
fact that table
SELECT COUNT(...)
may perform
dirty reads, and is not a bug in the
NDB
storage engine. (See Bug
#31321 for more information.)
Starting, stopping, or restarting a node may give rise to temporary errors causing some transactions to fail. These include the following cases:
Temporary errors. When first starting a node, it is possible that you may see Error 1204 Temporary failure, distribution changed and similar temporary errors.
Errors due to node failure. The stopping or failure of any data node can result in a number of different node failure errors. (However, there should be no aborted transactions when performing a planned shutdown of the cluster.)
In either of these cases, any errors that are generated must be handled within the application. This should be done by retrying the transaction.
See also Section 17.1.6.2, “Limits and Differences of MySQL Cluster from Standard MySQL Limits”.
Some database objects such as tables and indexes have different
limitations when using the
NDBCLUSTER
storage engine:
Table names containing special characters.
NDB
tables whose names
contain characters other than letters, numbers, dashes,
and underscores and which are created on one SQL node were
not always discovered correctly by other SQL nodes. (Bug
#31470)
This issue was fixed in MySQL 5.1.23, MySQL Cluster NDB 6.2.7, and MySQL Cluster NDB 6.3.4.
Number of database objects.
The maximum number of all
NDB
database objects in a
single MySQL Cluster—including databases, tables,
and indexes—is limited to 20320.
Attributes per table. Prior to MySQL Cluster NDB 7.0.15 and MySQL Cluster NDB 7.1.4, the maximum number of attributes (that is, columns and indexes) per table is limited to 128.
Beginning with MySQL Cluster NDB 7.0.15 and MySQL Cluster NDB 7.1.4, this limit is increased to 512.
Attributes per key. The maximum number of attributes per key is 32.
Row size.
Prior to MySQL Cluster NDB 7.0, the maximum permitted size
of any one row was 8052 bytes; in MySQL Cluster NDB 7.0
and later, this is 14000 bytes. Each
BLOB
or
TEXT
column contributes 256
+ 8 = 264 bytes to this total.
A number of features supported by other storage engines are not
supported for NDB
tables. Trying to
use any of these features in MySQL Cluster does not cause errors
in or of itself; however, errors may occur in applications that
expects the features to be supported or enforced:
Foreign key constraints.
Prior to MySQL Cluster NDB 7.3, the foreign key construct
is ignored, just as it is by MyISAM
tables. Foreign keys are supported in MySQL Cluster NDB
7.3 and later.
Index prefixes.
Prefixes on indexes are not supported for
NDBCLUSTER
tables. If a prefix is used
as part of an index specification in a statement such as
CREATE TABLE
,
ALTER TABLE
, or
CREATE INDEX
, the prefix is
ignored.
OPTIMIZE operations.
OPTIMIZE
operations are not supported.
Beginning with MySQL Cluster NDB 6.3.7, this limitation has been lifted. See Section 17.1.6.11, “Previous MySQL Cluster Issues Resolved in MySQL 5.1, MySQL Cluster NDB 6.x, and MySQL Cluster NDB 7.x”, for more information.
LOAD TABLE ... FROM MASTER.
LOAD TABLE FROM MASTER
is
not supported.
Savepoints and rollbacks.
Savepoints and rollbacks to savepoints are ignored as in
MyISAM
.
Durability of commits. There are no durable commits on disk. Commits are replicated, but there is no guarantee that logs are flushed to disk on commit.
Replication.
Statement-based replication is not supported. Use
--binlog-format=ROW
(or
--binlog-format=MIXED
) when
setting up cluster replication. See
Section 17.6, “MySQL Cluster Replication”, for more
information.
See Section 17.1.6.3, “Limits Relating to Transaction Handling in MySQL Cluster”,
for more information relating to limitations on transaction
handling in NDB
.
The following performance issues are specific to or especially pronounced in MySQL Cluster:
Range scans.
There are query performance issues due to sequential
access to the NDB
storage
engine; it is also relatively more expensive to do many
range scans than it is with either
MyISAM
or InnoDB
.
Reliability of Records in range.
The Records in range
statistic is
available but is not completely tested or officially
supported. This may result in nonoptimal query plans in
some cases. If necessary, you can employ USE
INDEX
or FORCE INDEX
to alter
the execution plan. See Section 13.2.8.3, “Index Hint Syntax”, for
more information on how to do this.
Unique hash indexes.
Unique hash indexes created with USING
HASH
cannot be used for accessing a table if
NULL
is given as part of the key.
The following are limitations specific to the
NDBCLUSTER
storage engine:
Machine architecture. All machines used in the cluster must have the same architecture. That is, all machines hosting nodes must be either big-endian or little-endian, and you cannot use a mixture of both. For example, you cannot have a management node running on a PowerPC which directs a data node that is running on an x86 machine. This restriction does not apply to machines simply running mysql or other clients that may be accessing the cluster's SQL nodes.
Binary logging. MySQL Cluster has the following limitations or restrictions with regard to binary logging:
sql_log_bin
has no
effect on data operations; however, it is supported for
schema operations.
MySQL Cluster cannot produce a binary log for tables
having BLOB
columns but
no primary key.
Only the following schema operations are logged in a cluster binary log which is not on the mysqld executing the statement:
See also Section 17.1.6.10, “Limitations Relating to Multiple MySQL Cluster Nodes”.
Disk Data object maximums and minimums. Disk data objects are subject to the following maximums and minimums:
Maximum number of tablespaces: 232 (4294967296)
Maximum number of data files per tablespace: 216 (65536)
Maximum data file size: The theoretical limit is 64G; however, in MySQL 5.1 (including MySQL Cluster NDB 6.X and 7.X through 7.1), the practical upper limit is 32G. This is equivalent to 32768 extents of 1M each.
Since a MySQL Cluster Disk Data table can use at most 1
tablespace, this means that the theoretical upper limit to
the amount of data (in bytes) that can be stored on disk by
a single NDB
table is 32G * 65536 =
2251799813685248, or approximately 2 petabytes.
The theoretical maximum number of extents per tablespace data file is 216 (65536); however, for practical purposes, the recommended maximum number of extents per data file is 215 (32768).
The minimum and maximum possible sizes of extents for tablespace data files are 32K and 2G, respectively. See Section 13.1.18, “CREATE TABLESPACE Syntax”, for more information.
Disk Data tables and diskless mode. Use of Disk Data tables is not supported when running the cluster in diskless mode. Beginning with MySQL 5.1.12, it is prohibited altogether. (Bug #20008)
Multiple SQL nodes.
The following are issues relating to the use of multiple MySQL
servers as MySQL Cluster SQL nodes, and are specific to the
NDBCLUSTER
storage engine:
No distributed table locks.
A LOCK TABLES
works only
for the SQL node on which the lock is issued; no other SQL
node in the cluster “sees” this lock. This is
also true for a lock issued by any statement that locks
tables as part of its operations. (See next item for an
example.)
ALTER TABLE operations.
ALTER TABLE
is not fully
locking when running multiple MySQL servers (SQL nodes).
(As discussed in the previous item, MySQL Cluster does not
support distributed table locks.)
Multiple management nodes. When using multiple management servers:
You must give nodes explicit IDs in connect strings because automatic allocation of node IDs does not work across multiple management servers.
In MySQL Cluster NDB 7.0.7 and earlier, you must take extreme care to have the same configurations for all management servers; no special checks for consistency are performed.
Beginning with MySQL Cluster NDB 7.0.8, a management server
when first starting checks for any other management server
in the same MySQL Cluster, and upon successful connection to
the other management server uses its configuration data.
This means that the management server
--reload
and
--initial
startup options
are ignored unless the management server is the only one
running. It also means that, when performing a rolling
restart of a MySQL Cluster with multiple management nodes,
the management server reads its own configuration file if
(and only if) it is the only management server running in
this MySQL Cluster. See
Section 17.5.5, “Performing a Rolling Restart of a MySQL Cluster”, for more
information.
Multiple network addresses. Multiple network addresses per data node are not supported. Use of these is liable to cause problems: In the event of a data node failure, an SQL node waits for confirmation that the data node went down but never receives it because another route to that data node remains open. This can effectively make the cluster inoperable.
It is possible to use multiple network hardware
interfaces (such as Ethernet cards) for a
single data node, but these must be bound to the same address.
This also means that it not possible to use more than one
[tcp]
section per connection in the
config.ini
file. See
Section 17.3.2.8, “MySQL Cluster TCP/IP Connections”, for more
information.
A number of limitations and related issues existing in earlier versions of MySQL Cluster have been resolved:
Variable-length column support.
The NDBCLUSTER
storage engine
now supports variable-length column types for in-memory
tables.
Previously, for example, any Cluster table having one or
more VARCHAR
fields which
contained only relatively small values, much more memory and
disk space were required when using the
NDBCLUSTER
storage engine than
would have been the case for the same table and data using
the MyISAM
engine. In other words, in the
case of a VARCHAR
column,
such a column required the same amount of storage as a
CHAR
column of the same size.
In MySQL 5.1, this is no longer the case for in-memory
tables, where storage requirements for variable-length
column types such as VARCHAR
and BINARY
are comparable to those for
these column types when used in MyISAM
tables (see Section 11.7, “Data Type Storage Requirements”).
For MySQL Cluster Disk Data tables, the fixed-width limitation continues to apply. See Section 17.5.12, “MySQL Cluster Disk Data Tables”.
Replication with MySQL Cluster. It is now possible to use MySQL replication with Cluster databases. For details, see Section 17.6, “MySQL Cluster Replication”.
Circular Replication. Circular replication is also supported with MySQL Cluster, beginning with MySQL 5.1.18. See Section 17.6.10, “MySQL Cluster Replication: Multi-Master and Circular Replication”.
auto_increment_increment and auto_increment_offset.
The
auto_increment_increment
and auto_increment_offset
server system variables are supported for Cluster
replication beginning with MySQL 5.1.20, MySQL Cluster NDB
6.2.5, and MySQL Cluster 6.3.2.
Database autodiscovery and online schema changes.
Autodiscovery of databases is now supported for multiple
MySQL servers accessing the same MySQL Cluster. Formerly,
autodiscovery in MySQL Cluster 5.1 and MySQL Cluster NDB
6.x releases required that a given
mysqld was already running and
connected to the cluster at the time that the database was
created on a different mysqld—in
other words, when a mysqld process
connected to the cluster after a database named
db_name
was created, it was
necessary to issue a CREATE DATABASE
or
db_name
CREATE SCHEMA
statement on
the “new” MySQL server when it first accessed
that MySQL Cluster. Beginning with MySQL Cluster NDB
6.2.16 and MySQL Cluster NDB 6.3.18, such a
db_name
CREATE
statement is no longer required.
(Bug #39612)
This also means that online schema changes in
NDB
tables are now possible.
That is, the result of operations such as
ALTER TABLE
and
CREATE INDEX
performed on one
SQL node in the cluster are now visible to the cluster's
other SQL nodes without any additional action being taken.
Backup and restore between architectures. Beginning with MySQL 5.1.10, it is possible to perform a Cluster backup and restore between different architectures. Previously—for example—you could not back up a cluster running on a big-endian platform and then restore from that backup to a cluster running on a little-endian system. (Bug #19255)
Character set directory.
Beginning with MySQL 5.1.10, it is possible to install
MySQL with Cluster support to a nondefault location and
change the search path for font description files using
either the --basedir
or
--character-sets-dir
options. (Previously, ndbd in MySQL 5.1
searched only the default path—typically
/usr/local/mysql/share/mysql/charsets
—for
character sets.)
Multiple management servers. In MySQL 5.1 (including all MySQL Cluster NDB 6.x and later versions), it is no longer necessary, when running multiple management servers, to restart all the cluster's data nodes to enable the management nodes to see one another.
Also, when using multiple management servers and starting concurrently several API nodes (possibly including one or more SQL nodes) whose connection strings listed the management servers in different order, it was possible for 2 API nodes to be assigned the same node ID. This issue is resolved in MySQL Cluster NDB 6.2.17, 6.3.23, and 6.4.3. (Bug #42973)
Multiple data node processes per host. Beginning with MySQL Cluster NDB 6.2.0, you can use multiple data node processes on a single host. (In MySQL Cluster NDB 6.1, MySQL 5.1, and earlier release series, we did not support production MySQL Cluster deployments in which more than one ndbd process was run on a single physical machine.)
In addition, MySQL Cluster NDB 7.0 introduces support for multi-threaded data nodes (ndbmtd). See Section 17.1.4.3, “MySQL Cluster Development in MySQL Cluster NDB 7.0”, and Section 17.4.3, “ndbmtd — The MySQL Cluster Data Node Daemon (Multi-Threaded)”, for more information.
Identifiers.
Formerly (in MySQL 5.0 and earlier), database names, table
names and attribute names could not be as long for
NDB
tables as tables using
other storage engines, because attribute names were
truncated internally. In MySQL 5.1 and later, names of
MySQL Cluster databases, tables, and table columns follow
the same rules regarding length as they do for any other
storage engine.
Length of CREATE TABLE statements.
CREATE TABLE
statements may
be no more than 4096 characters in length. This
limitation affects MySQL 5.1.6, 5.1.7, and 5.1.8
only. (See Bug #17813)
IGNORE and REPLACE functionality.
In MySQL 5.1.7 and earlier,
INSERT
IGNORE
,
UPDATE
IGNORE
, and
REPLACE
were supported only
for primary keys, but not for unique keys. It was possible
to work around this issue by removing the constraint, then
dropping the unique index, performing any inserts, and
then adding the unique index again.
This limitation was removed for
INSERT
IGNORE
and REPLACE
in MySQL 5.1.8. (See Bug #17431.)
AUTO_INCREMENT columns.
In MySQL 5.1.10 and earlier versions, the maximum number
of tables having AUTO_INCREMENT
columns—including those belonging to hidden primary
keys—was 2048.
This limitation was lifted in MySQL 5.1.11.
Maximum number of cluster nodes. Prior to MySQL Cluster NDB 6.1.1, the total maximum number of nodes in a MySQL Cluster was 63, including all SQL nodes (MySQL Servers), API nodes (applications accessing the cluster other than MySQL servers), data nodes, and management servers.
Starting with MySQL Cluster NDB 6.1.1, the total maximum number of nodes in a MySQL Cluster is 255, including all SQL nodes (MySQL Servers), API nodes (applications accessing the cluster other than MySQL servers), data nodes, and management servers. The total number of data nodes and management nodes beginning with this version is 63, of which up to 48 can be data nodes.
The limitation that a data node cannot have a node ID greater than 49 continues to apply.
Recovery of memory from deleted rows.
Beginning with MySQL Cluster NDB 6.3.7, memory can be
reclaimed from an NDB
table
for reuse with any NDB
table
by employing OPTIMIZE
TABLE
, subject to the following limitations:
Only in-memory tables are supported; the
OPTIMIZE TABLE
statement
still has no effect on MySQL Cluster Disk Data tables.
Only variable-length columns (such as those declared as
VARCHAR
,
TEXT
, or
BLOB
) are supported.
However, you can force columns defined using
fixed-length data types (such as
CHAR
) to be dynamic using
the ROW_FORMAT
or
COLUMN_FORMAT
option with a
CREATE TABLE
or
ALTER TABLE
statement.
See Section 13.1.17, “CREATE TABLE Syntax”, and Section 13.1.7, “ALTER TABLE Syntax”, for information on these options.
You can regulate the effects of OPTIMIZE
on performance by adjusting the value of the global system
variable ndb_optimization_delay
, which
sets the number of milliseconds to wait between batches of
rows being processed by OPTIMIZE
. The
default value is 10 milliseconds. It is possible to set a
lower value (to a minimum of 0
), but not
recommended. The maximum is 100000 milliseconds (that is,
100 seconds).
Implicit Rollbacks.
Prior to MySQL Cluster NDB 6.2.17 and MySQL Cluster NDB
6.3.19, MySQL Cluster did not automatically roll back a
transaction that was aborted by a duplicate key or similar
error, and subsequent statements raised ERROR
1296 (HY000): Got error 4350 'Transaction already aborted'
from NDBCLUSTER. In such cases, it was
necessary to issue an explicit
ROLLBACK
statement first, and then to retry the entire transaction.
Beginning with MySQL Cluster NDB 6.2.17 and MySQL Cluster NDB 6.3.19, this limitation has been removed; now, an error which causes a transaction to be aborted generates an implicit rollback of the entire transaction. This is logged with the warning Storage engine NDB does not support rollback for this statement. Transaction rolled back and must be restarted. A statement subsequent to this starts a new transaction. (Bug #32656)
The NDBCLUSTER
storage engine
does not support partial transactions or partial rollbacks
of transactions in any version of MySQL Cluster.
Number of tables.
Previously, the maximum number of
NDBCLUSTER
tables in a single
MySQL Cluster was 1792, but this is no longer the case in
MySQL 5.1 and later MySQL Cluster releases. However, the
number of tables is still included in the total maximum
number of NDBCLUSTER
database
objects (20320). (See
Section 17.1.6.5, “Limits Associated with Database Objects in MySQL Cluster”.)
DDL operations.
Beginning with MySQL Cluster NDB 6.4.0, DDL operations
(such as CREATE TABLE
or
ALTER TABLE
) are protected
from data node failures. Previously, if a data node failed
while trying to perform one of these, the data dictionary
became locked and no further DDL statements could be
executed without restarting the cluster (Bug #36718).
Adding and dropping of data nodes. In MySQL Cluster NDB 6.3 and previous versions of MySQL Cluster, the online adding or dropping of data nodes was not possible; such operations required a complete shutdown and restart of the entire cluster. In MySQL Cluster NDB 7.0 (beginning with MySQL Cluster NDB 6.4.0) and later MySQL Cluster release series, it is possible to add new data nodes to a running MySQL Cluster by performing a rolling restart, so that the cluster and the data stored in it remain available to applications.
When planning to increase the number of data nodes in the cluster online in MySQL Cluster NDB 7.0 or MySQL Cluster NDB 7.1, you should be aware of and take into account the following issues:
New data nodes can be added online to a MySQL Cluster only as part of a new node group.
New data nodes can be added online, but cannot yet be dropped online. Reducing the number of data nodes still requires a system restart of the cluster.
As in previous MySQL Cluster releases, it is not
possible to change online either the number of replicas
(NoOfReplicas
configuration parameter) or the number of data nodes per
node group. These changes require a system restart.
Redistribution of existing cluster data using the new data nodes is not automatic; however, this can be accomplished using simple SQL statements in the mysql client or other MySQL client application once the nodes have been added. During this procedure, it is not possible to perform DDL operations, although DML operations can continue as normal.
The distribution of new cluster data (that is, data stored in the cluster after the new nodes have been added) uses the new nodes without manual intervention.
For more information, see Section 17.5.13, “Adding MySQL Cluster Data Nodes Online”.
Native support for default column values.
Starting with MySQL Cluster NDB 7.1.0, default values for
table columns are stored by
NDBCLUSTER
, rather than by
the MySQL server as was previously the case. Because less
data must be sent from an SQL node to the data nodes,
inserts on tables having column value defaults can be
performed more efficiently than before.
Tables created using previous MySQL Cluster releases can
still be used in MySQL Cluster 7.1.0 and later, although
they do not support native default values and continue to
use defaults supplied by the MySQL server until they are
upgraded. This can be done by means of an offline
ALTER TABLE
statement.
You cannot set or change a table column's default
value using an online ALTER
TABLE
operation
InnoDB plugin support.
Previously, InnoDB
support in
MySQL Cluster was limited to the version built in to the
MySQl Server. Beginning with MySQL Cluster NDB 7.1.9,
MySQL Cluster also provides support for the
InnoDB
Plugin. See
Section 17.2, “MySQL Cluster Installation and Upgrades”, for
information about enabling
InnoDB
storage engine and
plugin support with MySQL Cluster.
Distribution of MySQL users and privileges.
Previously, MySQL users and privileges created on one SQL
node were unique to that SQL node, due to the fact that
the MySQL grant tables were restricted to using the
MyISAM
storage engine.
Beginning with MySQL Cluster NDB 7.2.0, it is possible,
following installation of the MySQL Cluster software and
setup of the desired users and privileges on one SQL node,
to convert the grant tables to use
NDB
and thus to distribute
the users and privileges across all SQL nodes connected to
the cluster. You can do this by loading and making use of
a set of stored procedures defined in an SQL script
supplied with the MySQL Cluster distribution. For more
information, see
Section 17.5.14, “Distributed MySQL Privileges for MySQL Cluster”.
Number of rows per partition. Previously, a single MySQL Cluster partition could hold a maximum of 46137488 rows. This limitation was removed in MySQL Cluster NDB 7.0.36 and MySQL Cluster NDB 7.1.25. (Bug #13844405, Bug #14000373)
If you are still using a previous MySQL Cluster release, you can work around this limitation by taking advantage of the fact that the number of partitions is the same as the number of data nodes in the cluster (see Section 17.1.2, “MySQL Cluster Nodes, Node Groups, Replicas, and Partitions”). This means that, by increasing the number of data nodes, you can increase the available space for storing data.
In MySQL Cluster NDB 7.0 and later, you can increase the number of data nodes in the cluster while the cluster remains in operation. See Section 17.5.13, “Adding MySQL Cluster Data Nodes Online”, for more information.
It is also possible to increase the number of partitions for
NDB
tables by using explicit
KEY
or LINEAR KEY
partitioning (see Section 18.2.4, “KEY Partitioning”).
This section describes the basics for planning, installing, configuring, and running a MySQL Cluster. Whereas the examples in Section 17.3, “Configuration of MySQL Cluster NDB 6.1-7.1” provide more in-depth information on a variety of clustering options and configuration, the result of following the guidelines and procedures outlined here should be a usable MySQL Cluster which meets the minimum requirements for availability and safeguarding of data.
For information about upgrading or downgrading a MySQL Cluster between release versions, see Section 17.2.7, “Upgrading and Downgrading MySQL Cluster”.
This section covers hardware and software requirements; networking issues; installation of MySQL Cluster; configuration issues; starting, stopping, and restarting the cluster; loading of a sample database; and performing queries.
Assumptions. The following sections make a number of assumptions regarding the cluster's physical and network configuration. These assumptions are discussed in the next few paragraphs.
Cluster nodes and host computers. The cluster consists of four nodes, each on a separate host computer, and each with a fixed network address on a typical Ethernet network as shown here:
Node | IP Address |
---|---|
Management node (mgmd) | 192.168.0.10 |
SQL node (mysqld) | 192.168.0.20 |
Data node "A" (ndbd) | 192.168.0.30 |
Data node "B" (ndbd) | 192.168.0.40 |
This may be made clearer by the following diagram:
Network addressing.
In the interest of simplicity (and reliability), this
How-To uses only numeric IP addresses.
However, if DNS resolution is available on your network, it is
possible to use host names in lieu of IP addresses in configuring
Cluster. Alternatively, you can use the hosts
file (typically /etc/hosts
for Linux and
other Unix-like operating systems,
C:\WINDOWS\system32\drivers\etc\hosts
on
Windows, or your operating system's equivalent) for providing
a means to do host lookup if such is available.
Potential hosts file issues.
A common problem when trying to use host names for Cluster nodes
arises because of the way in which some operating systems
(including some Linux distributions) set up the system's own
host name in the /etc/hosts
during
installation. Consider two machines with the host names
ndb1
and ndb2
, both in the
cluster
network domain. Red Hat Linux
(including some derivatives such as CentOS and Fedora) places the
following entries in these machines'
/etc/hosts
files:
# ndb1 /etc/hosts
:
127.0.0.1 ndb1.cluster ndb1 localhost.localdomain localhost
# ndb2 /etc/hosts
:
127.0.0.1 ndb2.cluster ndb2 localhost.localdomain localhost
SUSE Linux (including OpenSUSE) places these entries in the
machines' /etc/hosts
files:
# ndb1 /etc/hosts
:
127.0.0.1 localhost
127.0.0.2 ndb1.cluster ndb1
# ndb2 /etc/hosts
:
127.0.0.1 localhost
127.0.0.2 ndb2.cluster ndb2
In both instances, ndb1
routes
ndb1.cluster
to a loopback IP address, but gets a
public IP address from DNS for ndb2.cluster
,
while ndb2
routes ndb2.cluster
to a loopback address and obtains a public address for
ndb1.cluster
. The result is that each data node
connects to the management server, but cannot tell when any other
data nodes have connected, and so the data nodes appear to hang
while starting.
You cannot mix localhost
and other host names
or IP addresses in config.ini
. For these
reasons, the solution in such cases (other than to use IP
addresses for all
config.ini
HostName
entries) is to remove the fully qualified host names from
/etc/hosts
and use these in
config.ini
for all cluster hosts.
Host computer type. Each host computer in our installation scenario is an Intel-based desktop PC running a supported operating system installed to disk in a standard configuration, and running no unnecessary services. The core operating system with standard TCP/IP networking capabilities should be sufficient. Also for the sake of simplicity, we also assume that the file systems on all hosts are set up identically. In the event that they are not, you should adapt these instructions accordingly.
Network hardware. Standard 100 Mbps or 1 gigabit Ethernet cards are installed on each machine, along with the proper drivers for the cards, and that all four hosts are connected through a standard-issue Ethernet networking appliance such as a switch. (All machines should use network cards with the same throughout. That is, all four machines in the cluster should have 100 Mbps cards or all four machines should have 1 Gbps cards.) MySQL Cluster works in a 100 Mbps network; however, gigabit Ethernet provides better performance.
MySQL Cluster is not intended for use in a network for which throughput is less than 100 Mbps or which experiences a high degree of latency. For this reason (among others), attempting to run a MySQL Cluster over a wide area network such as the Internet is not likely to be successful, and is not supported in production.
Sample data.
We use the world
database which is available
for download from the MySQL Web site (see
http://dev.mysql.com/doc/index-other.html). We assume that
each machine has sufficient memory for running the operating
system, required MySQL Cluster processes, and (on the data nodes)
storing the database.
For general information about installing MySQL, see Chapter 2, Installing and Upgrading MySQL. For information about installation of MySQL Cluster on Linux and other Unix-like operating systems, see Section 17.2.1, “Installing MySQL Cluster on Linux”. For information about installation of MySQL Cluster on Windows operating systems, see Section 17.2.2, “Installing MySQL Cluster on Windows”.
For general information about MySQL Cluster hardware, software, and networking requirements, see Section 17.1.3, “MySQL Cluster Hardware, Software, and Networking Requirements”.
This section covers installation of MySQL Cluster on Linux and other Unix-like operating systems. While the next few sections refer to a Linux operating system, the instructions and procedures given there should be easily adaptable to other supported Unix-like platforms.
Beginning with MySQL Cluster NDB 7.1.3, MySQL Cluster is also supported for production use on Windows operating systems; for installation and setup instructions specific to Windows, see Section 17.2.2, “Installing MySQL Cluster on Windows”.
Each MySQL Cluster host computer must have the correct executable programs installed. A host running an SQL node must have installed on it a MySQL Server binary (mysqld). Management nodes require the management server daemon (ndb_mgmd); data nodes require the data node daemon (ndbd; in MySQL Cluster NDB 7.0 and later, you can use ndbmtd instead. It is not necessary to install the MySQL Server binary on management node hosts and data node hosts. It is recommended that you also install the management client (ndb_mgm) on the management server host.
Installation of MySQL Cluster on Linux can be done using precompiled binaries from Oracle (downloaded as a .tar.gz archive), with RPM packages (also available from Oracle), or from source code. All three of these installation methods are described in the section that follow.
Regardless of the method used, it is still necessary following installation of the MySQL Cluster binaries to create configuration files for all cluster nodes, before you can start the cluster. See Section 17.2.3, “Initial Configuration of MySQL Cluster”.
This section covers the steps necessary to install the correct executables for each type of Cluster node from precompiled binaries supplied by Oracle.
For setting up a cluster using precompiled binaries, the first
step in the installation process for each cluster host is to
download the latest MySQL Cluster NDB 6.3, MySQL Cluster NDB
7.0, or MySQL Cluster NDB 7.1 binary archive
(mysql-cluster-gpl-6.3.55-linux-i686-glibc23.tar.gz
,
mysql-cluster-gpl-7.0.42-linux-i686-glibc23.tar.gz
,
or
mysql-cluster-gpl-7.1.34-linux-i686-glibc23.tar.gz
,
respectively) from the
MySQL Cluster
downloads area. We assume that you have placed this file
in each machine's /var/tmp
directory. (If
you do require a custom binary, see
Section 2.11.3, “Installing MySQL Using a Development Source Tree”.)
After completing the installation, do not yet start any of the binaries. We show you how to do so following the configuration of the nodes (see Section 17.2.3, “Initial Configuration of MySQL Cluster”).
SQL nodes.
On each of the machines designated to host SQL nodes, perform
the following steps as the system root
user:
Check your /etc/passwd
and
/etc/group
files (or use whatever tools
are provided by your operating system for managing users and
groups) to see whether there is already a
mysql
group and mysql
user on the system. Some OS distributions create these as
part of the operating system installation process. If they
are not already present, create a new
mysql
user group, and then add a
mysql
user to this group:
shell>groupadd mysql
shell>useradd -g mysql mysql
The syntax for useradd and groupadd may differ slightly on different versions of Unix, or they may have different names such as adduser and addgroup.
Change location to the directory containing the downloaded
file, unpack the archive, and create a symbolic link named
mysql
to the mysql
directory. Note that the actual file and directory names
vary according to the MySQL Cluster version number.
shell>cd /var/tmp
shell>tar -C /usr/local -xzvf mysql-cluster-gpl-7.1.34-linux-i686-glibc23.tar.gz
shell>ln -s /usr/local/mysql-cluster-gpl-7.1.34-linux-i686-glibc23 /usr/local/mysql
Change location to the mysql
directory
and run the supplied script for creating the system
databases:
shell>cd mysql
shell>scripts/mysql_install_db --user=mysql
Set the necessary permissions for the MySQL server and data directories:
shell>chown -R root .
shell>chown -R mysql data
shell>chgrp -R mysql .
Copy the MySQL startup script to the appropriate directory, make it executable, and set it to start when the operating system is booted up:
shell>cp support-files/mysql.server /etc/rc.d/init.d/
shell>chmod +x /etc/rc.d/init.d/mysql.server
shell>chkconfig --add mysql.server
(The startup scripts directory may vary depending on your
operating system and version—for example, in some
Linux distributions, it is
/etc/init.d
.)
Here we use Red Hat's chkconfig for creating links to the startup scripts; use whatever means is appropriate for this purpose on your platform, such as update-rc.d on Debian.
Remember that the preceding steps must be repeated on each machine where an SQL node is to reside.
Data nodes.
Installation of the data nodes does not require the
mysqld binary. Only the MySQL Cluster data
node executable ndbd (single-threaded) or
ndbmtd (multi-threaded) is required. These
binaries can also be found in the .tar.gz
archive. Again, we assume that you have placed this archive in
/var/tmp
.
As system root
(that is, after using
sudo, su root, or your
system's equivalent for temporarily assuming the system
administrator account's privileges), perform the following steps
to install the data node binaries on the data node hosts:
Change location to the /var/tmp
directory, and extract the ndbd and
ndbmtd binaries from the archive into a
suitable directory such as
/usr/local/bin
:
shell>cd /var/tmp
shell>tar -zxvf mysql-5.1.73-ndb-7.1.34-linux-i686-glibc23.tar.gz
shell>cd mysql-5.1.73-ndb-7.1.34-linux-i686-glibc23
shell>cp bin/ndbd /usr/local/bin/ndbd
shell>cp bin/ndbmtd /usr/local/bin/ndbmtd
(You can safely delete the directory created by unpacking
the downloaded archive, and the files it contains, from
/var/tmp
once
ndb_mgm and ndb_mgmd
have been copied to the executables directory.)
Change location to the directory into which you copied the files, and then make both of them executable:
shell>cd /usr/local/bin
shell>chmod +x ndb*
The preceding steps should be repeated on each data node host.
Although only one of the data node executables is required to run a MySQL Cluster data node, we have shown you how to install both ndbd and ndbmtd in the preceding instructions. We recommend that you do this when installing or upgrading MySQL Cluster, even if you plan to use only one of them, since this will save time and trouble in the event that you later decide to change from one to the other.
The data directory on each machine hosting a data node is
/usr/local/mysql/data
. This piece of
information is essential when configuring the management node.
(See Section 17.2.3, “Initial Configuration of MySQL Cluster”.)
Management nodes.
Installation of the management node does not require the
mysqld binary. Only the MySQL Cluster
management server (ndb_mgmd) is required;
you most likely want to install the management client
(ndb_mgm) as well. Both of these binaries
also be found in the .tar.gz
archive.
Again, we assume that you have placed this archive in
/var/tmp
.
As system root
, perform the following steps
to install ndb_mgmd and
ndb_mgm on the management node host:
Change location to the /var/tmp
directory, and extract the ndb_mgm and
ndb_mgmd from the archive into a suitable
directory such as /usr/local/bin
:
shell>cd /var/tmp
shell>tar -zxvf mysql-5.1.73-ndb-7.1.34-linux-i686-glibc23.tar.gz
shell>cd mysql-5.1.73-ndb-7.1.34-linux-i686-glibc23
shell>cp bin/ndb_mgm* /usr/local/bin
(You can safely delete the directory created by unpacking
the downloaded archive, and the files it contains, from
/var/tmp
once
ndb_mgm and ndb_mgmd
have been copied to the executables directory.)
Change location to the directory into which you copied the files, and then make both of them executable:
shell>cd /usr/local/bin
shell>chmod +x ndb_mgm*
In Section 17.2.3, “Initial Configuration of MySQL Cluster”, we create configuration files for all of the nodes in our example MySQL Cluster.
This section covers the steps necessary to install the correct executables for each type of MySQL Cluster node using RPM packages supplied by Oracle.
RPMs are available for both 32-bit and 64-bit Linux platforms. For a MySQL Cluster, three RPMs are required:
The Server RPM (for
example,
MySQL-Cluster-gpl-server-6.3.55-0.sles10.i586.rpm
,
MySQL-Cluster-gpl-server-7.0.42-0.sles10.i586.rpm
,
or
MySQL-Cluster-gpl-server-7.1.34-0.sles10.i586.rpm
),
which supplies the core files needed to run a MySQL Server
with NDBCLUSTER
storage engine
support (that is, as a MySQL Cluster SQL node).
If you do not have your own client application capable of
administering a MySQL server, you should also obtain and
install the Client RPM (for
example,
MySQL-Cluster-gpl-client-6.3.55-0.sles10.i586.rpm
,
MySQL-Cluster-gpl-client-7.0.42-0.sles10.i586.rpm
,
or
MySQL-Cluster-gpl-client-7.1.34-0.sles10.i586.rpm
).
The Cluster storage engine
RPM (for example,
MySQL-Cluster-gpl-storage-6.3.55-0.sles10.i586.rpm
,
MySQL-Cluster-gpl-storage-7.0.42-0.sles10.i586.rpm
,
or
MySQL-Cluster-gpl-storage-7.1.34-0.sles10.i586.rpm
),
which supplies the MySQL Cluster data node binary
(ndbd).
The Cluster storage engine management
RPM (for example,
MySQL-Cluster-gpl-management-6.3.55-0.sles10.i586.rpm
,
MySQL-Cluster-gpl-management-7.0.42-0.sles10.i586.rpm
,
or
MySQL-Cluster-gpl-management-7.1.34-0.sles10.i586.rpm
)
which provides the MySQL Cluster management server binary
(ndb_mgmd).
In addition, you should also obtain the
NDB Cluster - Storage engine basic
tools RPM (for example,
MySQL-Cluster-gpl-tools-6.3.55-0.sles10.i586.rpm
,
MySQL-Cluster-gpl-tools-7.0.42-0.sles10.i586.rpm
,
or
MySQL-Cluster-gpl-tools-7.1.34-0.sles10.i586.rpm
),
which supplies several useful applications for working with a
MySQL Cluster. The most important of these is the MySQL Cluster
management client (ndb_mgm). The
NDB Cluster - Storage engine extra
tools RPM (for example,
MySQL-Cluster-gpl-extra-6.3.55-0.sles10.i586.rpm
,
MySQL-Cluster-gpl-extra-7.0.42-0.sles10.i586.rpm
,
or
MySQL-Cluster-gpl-extra-7.1.34-0.sles10.i586.rpm
)
contains some additional testing and monitoring programs, but is
not required to install a MySQL Cluster. (For more information
about these additional programs, see
Section 17.4, “MySQL Cluster Programs”.)
The MySQL Cluster version number in the RPM file names (shown
here as 6.3.55
,
7.0.42
, or
7.1.34
) can vary according
to the version which you are actually using. It is
very important that all of the Cluster RPMs to be installed have
the same version number. The glibc
version number (if present), and architecture designation (shown
here as i586
) should be appropriate to the
machine on which the RPM is to be installed.
Data nodes. On a computer that is to host a cluster data node it is necessary to install only the NDB Cluster - Storage engine RPM. To do so, copy this RPM to the data node host, and run the following command as the system root user, replacing the name shown for the RPM as necessary to match that of the RPM downloaded from the MySQL web site:
shell> rpm -Uhv MySQL-Cluster-gpl-storage-7.1.34-0.sles10.i586.rpm
The previous command installs the MySQL Cluster data node binary
(ndbd) in the /usr/sbin
directory.
SQL nodes. On each machine to be used for hosting a cluster SQL node, install the Server RPM by executing the following command as the system root user, replacing the name shown for the RPM as necessary to match the name of the RPM downloaded from the MySQL web site:
shell> rpm -Uhv MySQL-Cluster-gpl-server-7.1.34-0.sles10.i586.rpm
This installs the MySQL server binary
(mysqld) in the
/usr/sbin
directory, as well as all needed
MySQL Server support files. It also installs the
mysql.server and
mysqld_safe startup scripts in
/usr/share/mysql
and
/usr/bin
, respectively. The RPM installer
should take care of general configuration issues (such as
creating the mysql
user and group, if needed)
automatically.
To administer the SQL node (MySQL server), you should also install the Client RPM, as shown here:
shell> rpm -Uhv MySQL-Cluster-gpl-client-7.1.34-0.sles10.i586.rpm
This installs the mysql client program.
Management nodes. To install the MySQL Cluster management server, it is necessary only to use the NDB Cluster - Storage engine management RPM. Copy this RPM to the computer intended to host the management node, and then install it by running the following command as the system root user (replace the name shown for the RPM as necessary to match that of the Storage engine management RPM downloaded from the MySQL web site):
shell> rpm -Uhv MySQL-Cluster-gpl-management-7.1.34-0.sles10.i586.rpm
This installs the management server binary
(ndb_mgmd) to the
/usr/sbin
directory.
You should also install the NDB
management client, which is supplied by the
Storage engine basic tools RPM.
Copy this RPM to the same computer as the management node, and
then install it by running the following command as the system
root user (again, replace the name shown for the RPM as
necessary to match that of the Storage
engine basic tools RPM downloaded from the MySQL web
site):
shell> rpm -Uhv MySQL-Cluster-gpl-tools-7.1.34-0.sles10.i586.rpm
The Storage engine basic tools
RPM installs the MySQL Cluster management client
(ndb_mgm) to the
/usr/bin
directory.
You can also install the Cluster storage engine extra tools RPM, if you wish, as shown here:
shell> rpm -Uhv MySQL-Cluster-gpl-extra-7.1.34-0.sles10.i586.rpm
You may find the extra tools useful; however the Cluster storage engine extra tools RPM is not required to install a working MySQL Cluster.
See Section 2.5.1, “Installing MySQL on Linux Using RPM Packages”, for general information about installing MySQL using RPMs supplied by Oracle.
After installing from RPM, you still need to configure the cluster as discussed in Section 17.2.3, “Initial Configuration of MySQL Cluster”.
This section provides information about compiling MySQL Cluster on Linux and other Unix-like platforms. Building MySQL Cluster from source is similar to building the standard MySQL Server, although it differs in a few key respects discussed here. For general information about building MySQL from source, see Section 2.11, “Installing MySQL from Source”. Beginning with MySQL Cluster NDB 7.1.3, MySQL Cluster is also supported on Windows platforms, and can be built on Windows from source. For information about compiling MySQL Cluster on Windows platforms, see Section 17.2.2.2, “Compiling and Installing MySQL Cluster from Source on Windows”.
Building MySQL Cluster requires using the MySQL Cluster sources.
These are available from the MySQL Cluster downloads page at
http://dev.mysql.com/downloads/cluster/. The archived source
file should have a name similar to
mysql-cluster-gpl-6.3.55.tar.gz
,
mysql-cluster-gpl-7.0.42.tar.gz
,
or
mysql-cluster-gpl-7.1.34.tar.gz
.
You can also obtain MySQL development sources from
launchpad.net. Attempting to build MySQL Cluster from standard
MySQL 5.1 sources is not supported.
In addition to any other configure options
you wish to use, be sure to include
--with-plugins=ndbcluster
,
--with-plugins=max
, or , or
--with-plugins=max-no-innodb
.
Either of these options causes the binaries for the management
nodes, data nodes, and other MySQL Cluster programs to be built;
it also causes mysqld to be compiled with
NDB
storage engine support.
After you have run make && make install (or your system's equivalent), the result is similar to what is obtained by unpacking a precompiled binary to the same location. However, the layout can differ. These differences are covered in the next few paragraphs.
Prior to MySQL Cluster NDB 7.1.9, MySQL Cluster was not
compatible with the InnoDB
Plugin;
in earlier MySQL Cluster releases, only the version of
InnoDB
supplied with the MySQL
Server could be used. Beginning with MySQL Cluster NDB 7.1.9,
you can build MySQL Cluster with
InnoDB
storage engine or plugin
support using the appropriate
--with-plugins
option for
configure.
Management nodes.
When building from source and running the default
make install, the management server binary
(ndb_mgmd) is placed in
/usr/local/mysql/libexec
, while the
management client binary (ndb_mgm) can be
found in /usr/local/mysql/bin
. Only
ndb_mgmd is required to be present on a
management node host; however, it is also a good idea to have
ndb_mgm present on the same host machine.
Neither of these executables requires a specific location on
the host machine's file system.
Data nodes.
The only executable required on a data node host is
ndbd (mysqld, for
example, does not have to be present on the host machine). By
default when doing a source build, this file is placed in the
directory /usr/local/mysql/libexec
. For
installing on multiple data node hosts, only
ndbd need be copied to the other host
machine or machines. (This assumes that all data node hosts
use the same architecture and operating system; otherwise you
may need to compile separately for each different platform.)
ndbd need not be in any particular location
on the host's file system, as long as the location is known.
When compiling MySQL Cluster NDB 7.0 or later from source, no
special options are required for building multi-threaded data
node binaries. On Unix platforms, configuring the build with any
of the options --with-plugins=ndbcluster
,
--with-plugins=max
, or
--with-plugins=max-no-innodb
causes ndbmtd to be built automatically;
make install places the
ndbmtd binary in the
libexec
directory along with
mysqld, ndbd, and
ndb_mgm.
SQL nodes.
If you compile MySQL with clustering support, and perform the
default installation (using make install as
the system root
user),
mysqld is placed in
/usr/local/mysql/bin
. Follow the steps
given in Section 2.11, “Installing MySQL from Source” to make
mysqld ready for use. If you want to run
multiple SQL nodes, you can use a copy of the same
mysqld executable and its associated
support files on several machines. The easiest way to do this
is to copy the entire /usr/local/mysql
directory and all directories and files contained within it to
the other SQL node host or hosts, then repeat the steps from
Section 2.11, “Installing MySQL from Source” on each machine. If you
configure the build with a nondefault
--prefix
, you need to adjust
the directory accordingly.
In Section 17.2.3, “Initial Configuration of MySQL Cluster”, we create configuration files for all of the nodes in our example MySQL Cluster.
Experimental support for MySQL Cluster on Microsoft Windows operating systems was introduced in MySQL Cluster NDB 7.0. Beginning with MySQL Cluster NDB 7.1.3, production support is provided for MySQL Cluster on Windows, and MySQL Cluster binaries for Windows can be obtained from http://dev.mysql.com/downloads/cluster/. For information about installing MySQL Cluster on Windows from a binary release provided by Oracle, see Section 17.2.2.1, “Installing MySQL Cluster on Windows from a Binary Release”.
It is also possible to compile and install MySQL Cluster from source on Windows using Microsoft Visual Studio. For more information, see Section 17.2.2.2, “Compiling and Installing MySQL Cluster from Source on Windows”.
This section describes a basic installation of MySQL Cluster on
Windows using a binary no-install
MySQL
Cluster release provided by Oracle, using the same 4-node setup
outlined in the beginning of this section (see
Section 17.2, “MySQL Cluster Installation and Upgrades”), as shown in the
following table:
Node | IP Address |
---|---|
Management (MGMD) node | 192.168.0.10 |
MySQL server (SQL) node | 192.168.0.20 |
Data (NDBD) node "A" | 192.168.0.30 |
Data (NDBD) node "B" | 192.168.0.40 |
As on other platforms, the MySQL Cluster host computer running an SQL node must have installed on it a MySQL Server binary (mysqld.exe). You should also have the MySQL client (mysql.exe) on this host. For management nodes and data nodes, it is not necessary to install the MySQL Server binary; however, each management node requires the management server daemon (ndb_mgmd.exe); each data node requires the data node daemon (ndbd.exe or ndbmtd.exe). For this example, we refer to ndbd.exe as the data node executable, but you can install ndbmtd.exe, the multi-threaded version of this program, instead, in exactly the same way. You should also install the management client (ndb_mgm.exe) on the management server host. This section covers the steps necessary to install the correct Windows binaries for each type of MySQL Cluster node.
ndbmtd.exe was not included in MySQL Cluster NDB 7.1.3 binary releases for Windows, due to a problem with make_win_bin_dist. This issue was corrected in MySQL Cluster NDB 7.1.5.
As with other Windows programs, MySQL Cluster executables are
named with the .exe
file extension.
However, it is not necessary to include the
.exe
extension when invoking these
programs from the command line. Therefore, we often simply
refer to these programs in this documentation as
mysqld, mysql,
ndb_mgmd, and so on. You should understand
that, whether we refer (for example) to
mysqld or mysqld.exe,
either name means the same thing (the MySQL Server program).
For setting up a MySQL Cluster using Oracles's
no-install
binaries, the first step in the
installation process is to download the latest MySQL Cluster
Windows binary archive from
http://dev.mysql.com/downloads/cluster/. This archive has a
filename of the form
mysql-cluster-gpl-noinstall-
,
where ver
-winarch
.zipver
is the
NDB
storage engine version (such as
7.1.3
), and arch
is the architecture (32
for 32-bit binaries,
and 64
for 64-bit binaries). For example, the
MySQL Cluster NDB 7.1.3 no-install
archive
for 32-bit Windows systems is named
mysql-cluster-gpl-noinstall-7.1.3-win32.zip
.
You can run 32-bit MySQL Cluster binaries on both 32-bit and 64-bit versions of Windows; however, 64-bit MySQL Cluster binaries can be used only on 64-bit versions of Windows. If you are using a 32-bit version of Windows on a computer that has a 64-bit CPU, then you must use the 32-bit MySQL Cluster binaries.
To minimize the number of files that need to be downloaded from the Internet or copied between machines, we start with the computer where you intend to run the SQL node.
SQL node.
We assume that you have placed a copy of the
no-install
archive in the directory
C:\Documents and
Settings\
on the computer having the IP
address 192.168.0.20, where
username
\My
Documents\Downloadsusername
is the name of the current
user. (You can obtain this name using ECHO
%USERNAME%
on the command line.) To install and run
MySQL Cluster executables as Windows services, this user
should be a member of the Administrators
group.
Extract all the files from the archive. The Extraction Wizard
integrated with Windows Explorer is adequate for this task. (If
you use a different archive program, be sure that it extracts
all files and directories from the archive, and that it
preserves the archive's directory structure.) When you are
asked for a destination directory, enter
C:\
, which causes the Extraction Wizard to
extract the archive to the directory
C:\mysql-cluster-gpl-noinstall-
.
Rename this directory to ver
-winarch
C:\mysql
.
It is possible to install the MySQL Cluster binaries to
directories other than C:\mysql\bin
;
however, if you do so, you must modify the paths shown in this
procedure accordingly. In particular, if the MySQL Server (SQL
node) binary is installed to a location other than
C:\mysql
or C:\Program
Files\MySQL\MySQL Server 5.1
, or if the
SQL node's data directory is in a location other than
C:\mysql\data
or C:\Program
Files\MySQL\MySQL Server 5.1\data
, extra
configuration options must be used on the command line or added
to the my.ini
or
my.cnf
file when starting the SQL node. For
more information about configuring a MySQL Server to run in a
nonstandard location, see
Section 2.3.6, “Installing MySQL on Microsoft Windows Using a noinstall Zip Archive”.
For a MySQL Server with MySQL Cluster support to run as part of
a MySQL Cluster, it must be started with the options
--ndbcluster
and
--ndb-connectstring
. While you
can specify these options on the command line, it is usually
more convenient to place them in an option file. To do this,
create a new text file in Notepad or another text editor. Enter
the following configuration information into this file:
[mysqld] # Options for mysqld process: ndbcluster # run NDB storage engine ndb-connectstring=192.168.0.10 # location of management server
You can add other options used by this MySQL Server if desired
(see Section 2.3.6.2, “Creating an Option File”), but the file
must contain the options shown, at a minimum. Save this file as
C:\mysql\my.ini
. This completes the
installation and setup for the SQL node.
Data nodes.
A MySQL Cluster data node on a Windows host requires only a
single executable, one of either ndbd.exe
or ndbmtd.exe. For this example, we assume
that you are using ndbd.exe, but the same
instructions apply when using ndbmtd.exe.
On each computer where you wish to run a data node (the
computers having the IP addresses 192.168.0.30 and
192.168.0.40), create the directories
C:\mysql
,
C:\mysql\bin
, and
C:\mysql\cluster-data
; then, on the
computer where you downloaded and extracted the
no-install
archive, locate
ndbd.exe
in the
C:\mysql\bin
directory. Copy this file to
the C:\mysql\bin
directory on each of the
two data node hosts.
To function as part of a MySQL Cluster, each data node must be
given the address or hostname of the management server. You can
supply this information on the command line using the
--ndb-connectstring
or
-c
option when starting each data node process.
However, it is usually preferable to put this information in an
option file. To do this, create a new text file in Notepad or
another text editor and enter the following text:
[mysql_cluster] # Options for data node process: ndb-connectstring=192.168.0.10 # location of management server
Save this file as C:\mysql\my.ini
on the
data node host. Create another text file containing the same
information and save it on as
C:mysql\my.ini
on the other data node host,
or copy the my.ini file from the first data node host to the
second one, making sure to place the copy in the second data
node's C:\mysql
directory. Both data
node hosts are now ready to be used in the MySQL Cluster, which
leaves only the management node to be installed and configured.
Management node.
The only executable program required on a computer used for
hosting a MySQL Cluster management node is the management
server program ndb_mgmd.exe. However, in
order to administer the MySQL Cluster once it has been
started, you should also install the MySQL Cluster management
client program ndb_mgm.exe on the same
machine as the management server. Locate these two programs on
the machine where you downloaded and extracted the
no-install
archive; this should be the
directory C:\mysql\bin
on the SQL node
host. Create the directory C:\mysql\bin
on the computer having the IP address 192.168.0.10, then copy
both programs to this directory.
You should now create two configuration files for use by
ndb_mgmd.exe
:
A local configuration file to supply configuration data specific to the management node itself. Typically, this file needs only to supply the location of the MySQL Cluster global configuration file (see item 2).
To create this file, start a new text file in Notepad or another text editor, and enter the following information:
[mysql_cluster] # Options for management node process config-file=C:/mysql/bin/config.ini
Save this file as the plaintext file
C:\mysql\bin\my.ini
.
A global configuration file from which the management node
can obtain configuration information governing the MySQL
Cluster as a whole. At a minimum, this file must contain a
section for each node in the MySQL Cluster, and the IP
addresses or hostnames for the management node and all data
nodes (HostName
configuration parameter).
It is also advisable to include the following additional
information:
The IP address or hostname of any SQL nodes
The data memory and index memory allocated to each data
node (DataMemory
and IndexMemory
configuration parameters)
The number of replicas, using the
NoOfReplicas
configuration parameter (see
Section 17.1.2, “MySQL Cluster Nodes, Node Groups, Replicas, and Partitions”)
The directory where each data node stores it data and
log file, and the directory where the management node
keeps its log files (in both cases, the
DataDir
configuration parameter)
Create a new text file using a text editor such as Notepad, and input the following information:
[ndbd default]
# Options affecting ndbd processes on all data nodes:
NoOfReplicas=2 # Number of replicas
DataDir=C:/mysql/cluster-data # Directory for each data node's data files
# Forward slashes used in directory path,
# rather than backslashes. This is correct;
# see Important note in text
DataMemory=80M # Memory allocated to data storage
IndexMemory=18M # Memory allocated to index storage
# For DataMemory and IndexMemory, we have used the
# default values. Since the "world" database takes up
# only about 500KB, this should be more than enough for
# this example Cluster setup.
[ndb_mgmd]
# Management process options:
HostName=192.168.0.10 # Hostname or IP address of management node
DataDir=C:/mysql/bin/cluster-logs # Directory for management node log files
[ndbd]
# Options for data node "A":
# (one [ndbd] section per data node)
HostName=192.168.0.30 # Hostname or IP address
[ndbd]
# Options for data node "B":
HostName=192.168.0.40 # Hostname or IP address
[mysqld]
# SQL node options:
HostName=192.168.0.20 # Hostname or IP address
Save this file as the plaintext file
C:\mysql\bin\config.ini
.
A single backslash character (\
) cannot be
used when specifying directory paths in program options or
configuration files used by MySQL Cluster on Windows. Instead,
you must either escape each backslash character with a second
backslash (\\
), or replace the backslash
with a forward slash character (/
). For
example, the following line from the
[ndb_mgmd]
section of a MySQL Cluster
config.ini
file does not work:
DataDir=C:\mysql\bin\cluster-logs
Instead, you may use either of the following:
DataDir=C:\\mysql\\bin\\cluster-logs # Escaped backslashes
DataDir=C:/mysql/bin/cluster-logs # Forward slashes
For reasons of brevity and legibility, we recommend that you use forward slashes in directory paths used in MySQL Cluster program options and configuration files on Windows.
Oracle provides precompiled MySQL Cluster binaries for Windows which should be adequate for most users. However, if you wish, it is also possible to compile MySQL Cluster for Windows from source code. The procedure for doing this is almost identical to the procedure used to compile the standard MySQL Server binaries for Windows, and uses the same tools. However, there are two major differences:
To build MySQL Cluster, you must use the MySQL Cluster sources, which you can obtain from http://dev.mysql.com/downloads/cluster/.
Attempting to build MySQL Cluster from the source code for the standard MySQL Server is likely not to be successful, and is not supported by Oracle.
You must configure the build using the
WITH_NDBCLUSTER_STORAGE_ENGINE
option in
addition to any other build options you wish to use before
creating the Visual Studio project files. Once you have run
configure.js with the desired options,
you can create the project files and build from them in the
same manner as you do when compiling the standard MySQL
Server. For more information, see
Section 2.11.7, “Installing MySQL from Source on Windows”.
Prior to MySQL Cluster NDB 7.1.9, MySQL Cluster was not
compatible with the InnoDB
Plugin;
in earlier MySQL Cluster releases, only the version of
InnoDB
supplied with the MySQL
Server could be used. Beginning with MySQL Cluster NDB 7.1.9,
you can build MySQL Cluster with
InnoDB
storage engine or plugin
support on Windows using
WITH_INNOBASE_STORAGE_ENGINE
with
configure.js-
Once the build process is complete, you can create a Zip archive
containing the compiled binaries by running
make_win_bin_dist. The MySQL Cluster binaries
can be found in the bin
directory of the
resulting archive, which is equivalent to the
no-install
archive, and which can be
installed and configured in the same manner. For basic
information about how to accomplish these tasks, see
Section 17.2.2.1, “Installing MySQL Cluster on Windows from a Binary Release”.
On Windows, beginning with MySQL Cluster NDB 7.0.11, using
WITH_NDBCLUSTER_STORAGE_ENGINE
with
configure.js causes
ndbmtd.exe to be built automatically, and to
be found in the bin
directory of the
archive created by make_win_bin_dist. (It was
not possible to build ndbmtd.exe on Windows
prior to MySQL Cluster NDB 7.0.11.)
Once the MySQL Cluster executables and needed configuration files are in place, performing an initial start of the cluster is simply a matter of starting the MySQL Cluster executables for all nodes in the cluster. Each cluster node process must be started separately, and on the host computer where it resides. The management node should be started first, followed by the data nodes, and then finally by any SQL nodes.
On the management node host, issue the following command from the command line to start the management node process:
C:\mysql\bin> ndb_mgmd
2010-06-23 07:53:34 [MgmtSrvr] INFO -- NDB Cluster Management Server. mysql-5.1.73-ndb-7.1.34
2010-06-23 07:53:34 [MgmtSrvr] INFO -- Reading cluster configuration from 'config.ini'
The management node process continues to print logging output to the console. This is normal, because the management node is not running as a Windows service. (If you have used MySQL Cluster on a Unix-like platform such as Linux, you may notice that the management node's default behavior in this regard on Windows is effectively the opposite of its behavior on Unix systems, where it runs by default as a Unix daemon process. This behavior is also true of MySQL Cluster data node processes running on Windows.) For this reason, do not close the window in which ndb_mgmd.exe is running; doing so kills the management node process. (See Section 17.2.2.4, “Installing MySQL Cluster Processes as Windows Services”, where we show how to install and run MySQL Cluster processes as Windows services.)
The required -f
option tells the management
node where to find the global configuration file
(config.ini
). The long form of this
option is --config-file
.
A MySQL Cluster management node caches the configuration
data that it reads from config.ini
;
once it has created a configuration cache, it ignores the
config.ini
file on subsequent starts
unless forced to do otherwise. This means that, if the
management node fails to start due to an error in this
file, you must make the management node re-read
config.ini
after you have corrected
any errors in it. You can do this by starting
ndb_mgmd.exe with the
--reload
or
--initial
option on the
command line. Either of these options works to refresh the
configuration cache.
It is not necessary or advisable to use either of these
options in the management node's
my.ini
file.
For additional information about options which can be used with ndb_mgmd, see Section 17.4.4, “ndb_mgmd — The MySQL Cluster Management Server Daemon”, as well as Section 17.4.26, “Options Common to MySQL Cluster Programs — Options Common to MySQL Cluster Programs”.
On each of the data node hosts, run the command shown here to start the data node processes:
C:\mysql\bin> ndbd
2010-06-23 07:53:46 [ndbd] INFO -- Configuration fetched from 'localhost:1186', generation: 1
In each case, the first line of output from the data node process should resemble what is shown in the preceding example, and is followed by additional lines of logging output. As with the management node process, this is normal, because the data node is not running as a Windows service. For this reason, do not close the console window in which the data node process is running; doing so kills ndbd.exe. (For more information, see Section 17.2.2.4, “Installing MySQL Cluster Processes as Windows Services”.)
Do not start the SQL node yet; it cannot connect to the
cluster until the data nodes have finished starting, which
may take some time. Instead, in a new console window on the
management node host, start the MySQL Cluster management
client ndb_mgm.exe, which should be in
C:\mysql\bin
on the management node
host. (Do not try to re-use the console window where
ndb_mgmd.exe is running by typing
CTRL+C, as this kills the
management node.) The resulting output should look like
this:
C:\mysql\bin> ndb_mgm
-- NDB Cluster -- Management Client --
ndb_mgm>
When the prompt ndb_mgm>
appears, this
indicates that the management client is ready to receive
MySQL Cluster management commands. You can observe the
status of the data nodes as they start by entering
ALL STATUS
at the management client
prompt. This command causes a running report of the data
nodes's startup sequence, which should look something
like this:
ndb_mgm> ALL STATUS
Connected to Management Server at: localhost:1186
Node 2: starting (Last completed phase 3) (mysql-5.1.73-ndb-7.1.34)
Node 3: starting (Last completed phase 3) (mysql-5.1.73-ndb-7.1.34)
Node 2: starting (Last completed phase 4) (mysql-5.1.73-ndb-7.1.34)
Node 3: starting (Last completed phase 4) (mysql-5.1.73-ndb-7.1.34)
Node 2: Started (version 7.1.34)
Node 3: Started (version 7.1.34)
ndb_mgm>
Commands issued in the management client are not case-sensitive; we use uppercase as the canonical form of these commands, but you are not required to observe this convention when inputting them into the ndb_mgm client. For more information, see Section 17.5.2, “Commands in the MySQL Cluster Management Client”.
The output produced by ALL STATUS
is
likely to vary from what is shown here, according to the
speed at which the data nodes are able to start, the release
version number of the MySQL Cluster software you are using,
and other factors. What is significant is that, when you see
that both data nodes have started, you are ready to start
the SQL node.
You can leave ndb_mgm.exe running; it has no negative impact on the performance of the MySQL Cluster, and we use it in the next step to verify that the SQL node is connected to the cluster after you have started it.
On the computer designated as the SQL node host, open a
console window and navigate to the directory where you
unpacked the MySQL Cluster binaries (if you are following
our example, this is C:\mysql\bin
).
Start the SQL node by invoking mysqld.exe from the command line, as shown here:
C:\mysql\bin> mysqld --console
The --console
option causes
logging information to be written to the console, which can
be helpful in the event of problems. (Once you are satisfied
that the SQL node is running in a satisfactory manner, you
can stop it and restart it out without the
--console
option, so that
logging is performed normally.)
In the console window where the management client
(ndb_mgm.exe) is running on the
management node host, enter the SHOW
command, which should produce output similar to what is
shown here:
ndb_mgm> SHOW
Connected to Management Server at: localhost:1186
Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=2 @192.168.0.30 (Version: 5.1.73-ndb-7.1.34, Nodegroup: 0, *)
id=3 @192.168.0.40 (Version: 5.1.73-ndb-7.1.34, Nodegroup: 0)
[ndb_mgmd(MGM)] 1 node(s)
id=1 @192.168.0.10 (Version: 5.1.73-ndb-7.1.34)
[mysqld(API)] 1 node(s)
id=4 @192.168.0.20 (Version: 5.1.73-ndb-7.1.34)
You can also verify that the SQL node is connected to the
MySQL Cluster in the mysql client
(mysql.exe) using the
SHOW ENGINE NDB STATUS
statement.
You should now be ready to work with database objects and data
using MySQL Cluster's
NDBCLUSTER
storage engine. See
Section 17.2.5, “MySQL Cluster Example with Tables and Data”, for more
information and examples.
Beginning with MySQL Cluster NDB 7.0.16 and MySQL Cluster NDB 7.1.5, you can install ndb_mgmd.exe, ndbd.exe, and ndbmtd.exe as Windows services. For information on how to do this, see Section 17.2.2.4, “Installing MySQL Cluster Processes as Windows Services”).
Once you are satisfied that MySQL Cluster is running as desired, you can—beginning with MySQL Cluster NDB 7.0.16 and MySQL Cluster NDB 7.1.5—install the management nodes and data nodes as Windows services, so that these processes are started and stopped automatically whenever Windows is started or stopped. This also makes it possible to control these processes from the command line with the appropriate NET START or NET STOP command, or using the Windows graphical Services utility.
Installing programs as Windows services usually must be done using an account that has Administrator rights on the system.
To install the management node as a service on Windows, invoke
ndb_mgmd.exe from the command line on the
machine hosting the management node, using the
--install
option, as shown
here:
C:\> C:\mysql\bin\ndb_mgmd.exe --install
Installing service 'MySQL Cluster Management Server'
as '"C:\mysql\bin\ndbd.exe" "--service=ndb_mgmd"'
Service successfully installed.
When installing a MySQL Cluster program as a Windows service, you should always specify the complete path; otherwise the service installation may fail with the error The system cannot find the file specified.
The --install
option must be
used first, ahead of any other options that might be specified
for ndb_mgmd.exe. However, it is preferable
to specify such options in an options file instead. If your
options file is not in one of the default locations as shown in
the output of ndb_mgmd.exe
--help
, you can specify the
location using the
--config-file
option.
Now you should be able to start and stop the management server like this:
C:\>NET START ndb_mgmd
The MySQL Cluster Management Server service is starting. The MySQL Cluster Management Server service was started successfully. C:\>NET STOP ndb_mgmd
The MySQL Cluster Management Server service is stopping.. The MySQL Cluster Management Server service was stopped successfully.
You can also start or stop the management server as a Windows service using the descriptive name, as shown here:
C:\>NET START 'MySQL Cluster Management Server'
The MySQL Cluster Management Server service is starting. The MySQL Cluster Management Server service was started successfully. C:\>NET STOP 'MySQL Cluster Management Server'
The MySQL Cluster Management Server service is stopping.. The MySQL Cluster Management Server service was stopped successfully.
However, it is usually simpler to specify a short service name
or to permit the default service name to be used when installing
the service, and then reference that name when starting or
stopping the service. To specify a service name other than
ndb_mgmd
, append it to the
--install
option, as shown in
this example:
C:\> C:\mysql\bin\ndb_mgmd.exe --install=mgmd1
Installing service 'MySQL Cluster Management Server'
as '"C:\mysql\bin\ndb_mgmd.exe" "--service=mgmd1"'
Service successfully installed.
Now you should be able to start or stop the service using the name you have specified, like this:
C:\>NET START mgmd1
The MySQL Cluster Management Server service is starting. The MySQL Cluster Management Server service was started successfully. C:\>NET STOP mgmd1
The MySQL Cluster Management Server service is stopping.. The MySQL Cluster Management Server service was stopped successfully.
To remove the management node service, invoke
ndb_mgmd.exe with the
--remove
option, as shown here:
C:\> C:\mysql\bin\ndb_mgmd.exe --remove
Removing service 'MySQL Cluster Management Server'
Service successfully removed.
If you installed the service using a service name other than the
default, you can remove the service by passing this name as the
value of the --remove
option,
like this:
C:\> C:\mysql\bin\ndb_mgmd.exe --remove=mgmd1
Removing service 'mgmd1'
Service successfully removed.
Installation of a MySQL Cluster data node process as a Windows
service can be done in a similar fashion, using the
--install
option for
ndbd.exe (or ndbmtd.exe),
as shown here:
C:\> C:\mysql\bin\ndbd.exe --install
Installing service 'MySQL Cluster Data Node Daemon' as '"C:\mysql\bin\ndbd.exe" "--service=ndbd"'
Service successfully installed.
Now you can start or stop the data node using either the default service name or the descriptive name with net start or net stop, as shown in the following example:
C:\>NET START ndbd
The MySQL Cluster Data Node Daemon service is starting. The MySQL Cluster Data Node Daemon service was started successfully. C:\>NET STOP ndbd
The MySQL Cluster Data Node Daemon service is stopping.. The MySQL Cluster Data Node Daemon service was stopped successfully. C:\>NET START 'MySQL Cluster Data Node Daemon'
The MySQL Cluster Data Node Daemon service is starting. The MySQL Cluster Data Node Daemon service was started successfully. C:\>NET STOP 'MySQL Cluster Data Node Daemon'
The MySQL Cluster Data Node Daemon service is stopping.. The MySQL Cluster Data Node Daemon service was stopped successfully.
To remove the data node service, invoke
ndbd.exe with the
--remove
option, as shown here:
C:\> C:\mysql\bin\ndbd.exe --remove
Removing service 'MySQL Cluster Data Node Daemon'
Service successfully removed.
As with ndb_mgmd.exe (and
mysqld.exe), when installing
ndbd.exe as a Windows service, you can also
specify a name for the service as the value of
--install
, and then use it when
starting or stopping the service, like this:
C:\>C:\mysql\bin\ndbd.exe --install=dnode1
Installing service 'dnode1' as '"C:\mysql\bin\ndbd.exe" "--service=dnode1"' Service successfully installed. C:\>NET START dnode1
The MySQL Cluster Data Node Daemon service is starting. The MySQL Cluster Data Node Daemon service was started successfully. C:\>NET STOP dnode1
The MySQL Cluster Data Node Daemon service is stopping.. The MySQL Cluster Data Node Daemon service was stopped successfully.
If you specified a service name when installing the data node
service, you can use this name when removing it as well, by
passing it as the value of the
--remove
option, as shown here:
C:\> C:\mysql\bin\ndbd.exe --remove=dnode1
Removing service 'dnode1'
Service successfully removed.
Installation of the SQL node as a Windows service, starting the
service, stopping the service, and removing the service are done
in a similar fashion, using mysqld
--install
, NET START,
NET STOP, and mysqld
--remove
. For additional
information, see Section 2.3.6.7, “Starting MySQL Server as a Microsoft Windows Service”.
For our four-node, four-host MySQL Cluster, it is necessary to write four configuration files, one per node host.
Each data node or SQL node requires a
my.cnf
file that provides two pieces of
information: a connection string that
tells the node where to find the management node, and a line
telling the MySQL server on this host (the machine hosting the
data node) to enable the
NDBCLUSTER
storage engine.
For more information on connection strings, see Section 17.3.2.3, “MySQL Cluster Connection Strings”.
The management node needs a config.ini
file telling it how many replicas to maintain, how much memory
to allocate for data and indexes on each data node, where to
find the data nodes, where to save data to disk on each data
node, and where to find any SQL nodes.
Configuring the data nodes and SQL nodes.
The my.cnf
file needed for the data nodes
is fairly simple. The configuration file should be located in
the /etc
directory and can be edited using
any text editor. (Create the file if it does not exist.) For
example:
shell> vi /etc/my.cnf
We show vi being used here to create the file, but any text editor should work just as well.
For each data node and SQL node in our example setup,
my.cnf
should look like this:
[mysqld] # Options for mysqld process: ndbcluster # run NDB storage engine [mysql_cluster] # Options for MySQL Cluster processes: ndb-connectstring=192.168.0.10 # location of management server
After entering the preceding information, save this file and exit the text editor. Do this for the machines hosting data node “A”, data node “B”, and the SQL node.
Once you have started a mysqld process with
the ndbcluster
and
ndb-connectstring
parameters in the
[mysqld]
and
[mysql_cluster]
sections of the
my.cnf
file as shown previously, you cannot
execute any CREATE TABLE
or
ALTER TABLE
statements without
having actually started the cluster. Otherwise, these statements
will fail with an error. This is by design.
Configuring the management node.
The first step in configuring the management node is to create
the directory in which the configuration file can be found and
then to create the file itself. For example (running as
root
):
shell>mkdir /var/lib/mysql-cluster
shell>cd /var/lib/mysql-cluster
shell>vi config.ini
For our representative setup, the config.ini
file should read as follows:
[ndbd default] # Options affecting ndbd processes on all data nodes: NoOfReplicas=2 # Number of replicas DataMemory=80M # How much memory to allocate for data storage IndexMemory=18M # How much memory to allocate for index storage # For DataMemory and IndexMemory, we have used the # default values. Since the "world" database takes up # only about 500KB, this should be more than enough for # this example Cluster setup. [tcp default] # TCP/IP options: portnumber=2202 # This the default; however, you can use any # port that is free for all the hosts in the cluster # Note: It is recommended that you do not specify the port # number at all and simply allow the default value to be used # instead [ndb_mgmd] # Management process options: hostname=192.168.0.10 # Hostname or IP address of MGM node datadir=/var/lib/mysql-cluster # Directory for MGM node log files [ndbd] # Options for data node "A": # (one [ndbd] section per data node) hostname=192.168.0.30 # Hostname or IP address datadir=/usr/local/mysql/data # Directory for this data node's data files [ndbd] # Options for data node "B": hostname=192.168.0.40 # Hostname or IP address datadir=/usr/local/mysql/data # Directory for this data node's data files [mysqld] # SQL node options: hostname=192.168.0.20 # Hostname or IP address # (additional mysqld connections can be # specified for this node for various # purposes such as running ndb_restore)
The world
database can be downloaded from
http://dev.mysql.com/doc/, where it can be found listed
under “Examples”.
After all the configuration files have been created and these minimal options have been specified, you are ready to proceed with starting the cluster and verifying that all processes are running. We discuss how this is done in Section 17.2.4, “Initial Startup of MySQL Cluster”.
For more detailed information about the available MySQL Cluster configuration parameters and their uses, see Section 17.3.2, “MySQL Cluster Configuration Files”, and Section 17.3, “Configuration of MySQL Cluster NDB 6.1-7.1”. For configuration of MySQL Cluster as relates to making backups, see Section 17.5.3.3, “Configuration for MySQL Cluster Backups”.
The default port for Cluster management nodes is 1186; the default port for data nodes is 2202. However, the cluster can automatically allocate ports for data nodes from those that are already free.
Starting the cluster is not very difficult after it has been configured. Each cluster node process must be started separately, and on the host where it resides. The management node should be started first, followed by the data nodes, and then finally by any SQL nodes:
On the management host, issue the following command from the system shell to start the management node process:
shell> ndb_mgmd -f /var/lib/mysql-cluster/config.ini
ndb_mgmd must be told where to find its
configuration file, using the -f
or
--config-file
option. (See
Section 17.4.4, “ndb_mgmd — The MySQL Cluster Management Server Daemon”, for
details.)
For additional options which can be used with ndb_mgmd, see Section 17.4.26, “Options Common to MySQL Cluster Programs — Options Common to MySQL Cluster Programs”.
On each of the data node hosts, run this command to start the ndbd process:
shell> ndbd
If you used RPM files to install MySQL on the cluster host where the SQL node is to reside, you can (and should) use the supplied startup script to start the MySQL server process on the SQL node.
If all has gone well, and the cluster has been set up correctly, the cluster should now be operational. You can test this by invoking the ndb_mgm management node client. The output should look like that shown here, although you might see some slight differences in the output depending upon the exact version of MySQL that you are using:
shell>ndb_mgm
-- NDB Cluster -- Management Client -- ndb_mgm>SHOW
Connected to Management Server at: localhost:1186 Cluster Configuration --------------------- [ndbd(NDB)] 2 node(s) id=2 @192.168.0.30 (Version: 5.1.73-ndb-7.1.34, Nodegroup: 0, *) id=3 @192.168.0.40 (Version: 5.1.73-ndb-7.1.34, Nodegroup: 0) [ndb_mgmd(MGM)] 1 node(s) id=1 @192.168.0.10 (Version: 5.1.73-ndb-7.1.34) [mysqld(API)] 1 node(s) id=4 @192.168.0.20 (Version: 5.1.73-ndb-7.1.34)
The SQL node is referenced here as
[mysqld(API)]
, which reflects the fact that the
mysqld process is acting as a MySQL Cluster API
node.
The IP address shown for a given MySQL Cluster SQL or other API
node in the output of SHOW
is the address used by the SQL or API node to connect to the
cluster data nodes, and not to any management node.
You should now be ready to work with databases, tables, and data in MySQL Cluster. See Section 17.2.5, “MySQL Cluster Example with Tables and Data”, for a brief discussion.
The information in this section applies to MySQL Cluster running on both Unix and Windows platforms.
Working with database tables and data in MySQL Cluster is not much different from doing so in standard MySQL. There are two key points to keep in mind:
For a table to be replicated in the cluster, it must use the
NDBCLUSTER
storage engine. To
specify this, use the ENGINE=NDBCLUSTER
or
ENGINE=NDB
option when creating the table:
CREATE TABLEtbl_name
(col_name
column_definitions
) ENGINE=NDBCLUSTER;
Alternatively, for an existing table that uses a different
storage engine, use ALTER TABLE
to change the table to use
NDBCLUSTER
:
ALTER TABLE tbl_name
ENGINE=NDBCLUSTER;
Every NDBCLUSTER
table has a
primary key. If no primary key is defined by the user when a
table is created, the NDBCLUSTER
storage engine automatically generates a hidden one. Such a
key takes up space just as does any other table index. (It is
not uncommon to encounter problems due to insufficient memory
for accommodating these automatically created indexes.)
If you are importing tables from an existing database using the
output of mysqldump, you can open the SQL
script in a text editor and add the ENGINE
option to any table creation statements, or replace any existing
ENGINE
options. Suppose that you have the
world
sample database on another MySQL server
that does not support MySQL Cluster, and you want to export the
City
table:
shell> mysqldump --add-drop-table world City > city_table.sql
The resulting city_table.sql
file will
contain this table creation statement (and the
INSERT
statements necessary to
import the table data):
DROP TABLE IF EXISTS `City`;
CREATE TABLE `City` (
`ID` int(11) NOT NULL auto_increment,
`Name` char(35) NOT NULL default '',
`CountryCode` char(3) NOT NULL default '',
`District` char(20) NOT NULL default '',
`Population` int(11) NOT NULL default '0',
PRIMARY KEY (`ID`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
INSERT INTO `City` VALUES (1,'Kabul','AFG','Kabol',1780000);
INSERT INTO `City` VALUES (2,'Qandahar','AFG','Qandahar',237500);
INSERT INTO `City` VALUES (3,'Herat','AFG','Herat',186800);
(remaining INSERT statements omitted)
You need to make sure that MySQL uses the
NDBCLUSTER
storage engine for this
table. There are two ways that this can be accomplished. One of
these is to modify the table definition
before importing it into the Cluster
database. Using the City
table as an example,
modify the ENGINE
option of the definition as
follows:
DROP TABLE IF EXISTS `City`;
CREATE TABLE `City` (
`ID` int(11) NOT NULL auto_increment,
`Name` char(35) NOT NULL default '',
`CountryCode` char(3) NOT NULL default '',
`District` char(20) NOT NULL default '',
`Population` int(11) NOT NULL default '0',
PRIMARY KEY (`ID`)
) ENGINE=NDBCLUSTER DEFAULT CHARSET=latin1;
INSERT INTO `City` VALUES (1,'Kabul','AFG','Kabol',1780000);
INSERT INTO `City` VALUES (2,'Qandahar','AFG','Qandahar',237500);
INSERT INTO `City` VALUES (3,'Herat','AFG','Herat',186800);
(remaining INSERT statements omitted)
This must be done for the definition of each table that is to be
part of the clustered database. The easiest way to accomplish this
is to do a search-and-replace on the file that contains the
definitions and replace all instances of
TYPE=
or
engine_name
ENGINE=
with engine_name
ENGINE=NDBCLUSTER
. If you do not want to
modify the file, you can use the unmodified file to create the
tables, and then use ALTER TABLE
to
change their storage engine. The particulars are given later in
this section.
Assuming that you have already created a database named
world
on the SQL node of the cluster, you can
then use the mysql command-line client to read
city_table.sql
, and create and populate the
corresponding table in the usual manner:
shell> mysql world < city_table.sql
It is very important to keep in mind that the preceding command
must be executed on the host where the SQL node is running (in
this case, on the machine with the IP address
192.168.0.20
).
To create a copy of the entire world
database
on the SQL node, use mysqldump on the
noncluster server to export the database to a file named
world.sql
; for example, in the
/tmp
directory. Then modify the table
definitions as just described and import the file into the SQL
node of the cluster like this:
shell> mysql world < /tmp/world.sql
If you save the file to a different location, adjust the preceding instructions accordingly.
Running SELECT
queries on the SQL
node is no different from running them on any other instance of a
MySQL server. To run queries from the command line, you first need
to log in to the MySQL Monitor in the usual way (specify the
root
password at the Enter
password:
prompt):
shell> mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1 to server version: 5.1.73-ndb-7.1.34
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql>
We simply use the MySQL server's root
account and assume that you have followed the standard security
precautions for installing a MySQL server, including setting a
strong root
password. For more information, see
Section 2.12.2, “Securing the Initial MySQL Accounts”.
It is worth taking into account that Cluster nodes do
not make use of the MySQL privilege system
when accessing one another. Setting or changing MySQL user
accounts (including the root
account) effects
only applications that access the SQL node, not interaction
between nodes. See
Section 17.5.11.2, “MySQL Cluster and MySQL Privileges”, for
more information.
If you did not modify the ENGINE
clauses in the
table definitions prior to importing the SQL script, you should
run the following statements at this point:
mysql>USE world;
mysql>ALTER TABLE City ENGINE=NDBCLUSTER;
mysql>ALTER TABLE Country ENGINE=NDBCLUSTER;
mysql>ALTER TABLE CountryLanguage ENGINE=NDBCLUSTER;
Selecting a database and running a SELECT query against a table in that database is also accomplished in the usual manner, as is exiting the MySQL Monitor:
mysql>USE world;
mysql>SELECT Name, Population FROM City ORDER BY Population DESC LIMIT 5;
+-----------+------------+ | Name | Population | +-----------+------------+ | Bombay | 10500000 | | Seoul | 9981619 | | São Paulo | 9968485 | | Shanghai | 9696300 | | Jakarta | 9604900 | +-----------+------------+ 5 rows in set (0.34 sec) mysql>\q
Bye shell>
Applications that use MySQL can employ standard APIs to access
NDB
tables. It is important to
remember that your application must access the SQL node, and not
the management or data nodes. This brief example shows how we
might execute the SELECT
statement
just shown by using the PHP 5.X mysqli
extension running on a Web server elsewhere on the network:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type"
content="text/html; charset=iso-8859-1">
<title>SIMPLE mysqli SELECT</title>
</head>
<body>
<?php
# connect to SQL node:
$link = new mysqli('192.168.0.20', 'root', 'root_password
', 'world');
# parameters for mysqli constructor are:
# host, user, password, database
if( mysqli_connect_errno() )
die("Connect failed: " . mysqli_connect_error());
$query = "SELECT Name, Population
FROM City
ORDER BY Population DESC
LIMIT 5";
# if no errors...
if( $result = $link->query($query) )
{
?>
<table border="1" width="40%" cellpadding="4" cellspacing ="1">
<tbody>
<tr>
<th width="10%">City</th>
<th>Population</th>
</tr>
<?
# then display the results...
while($row = $result->fetch_object())
printf("<tr>\n <td align=\"center\">%s</td><td>%d</td>\n</tr>\n",
$row->Name, $row->Population);
?>
</tbody
</table>
<?
# ...and verify the number of rows that were retrieved
printf("<p>Affected rows: %d</p>\n", $link->affected_rows);
}
else
# otherwise, tell us what went wrong
echo mysqli_error();
# free the result set and the mysqli connection object
$result->close();
$link->close();
?>
</body>
</html>
We assume that the process running on the Web server can reach the IP address of the SQL node.
In a similar fashion, you can use the MySQL C API, Perl-DBI, Python-mysql, or MySQL Connectors to perform the tasks of data definition and manipulation just as you would normally with MySQL.
To shut down the cluster, enter the following command in a shell on the machine hosting the management node:
shell> ndb_mgm -e shutdown
The -e
option here is used to pass a command to
the ndb_mgm client from the shell. (See
Section 17.4.26, “Options Common to MySQL Cluster Programs — Options Common to MySQL Cluster Programs”, for more
information about this option.) The command causes the
ndb_mgm, ndb_mgmd, and any
ndbd or ndbmtd processes to
terminate gracefully. Any SQL nodes can be terminated using
mysqladmin shutdown and other means. On Windows
platforms, assuming that you have installed the SQL node as a
Windows service, you can use NET STOP MYSQL.
To restart the cluster on Unix platforms, run these commands:
On the management host (192.168.0.10
in our
example setup):
shell> ndb_mgmd -f /var/lib/mysql-cluster/config.ini
On each of the data node hosts
(192.168.0.30
and
192.168.0.40
):
shell> ndbd
Use the ndb_mgm client to verify that both data nodes have started successfully.
On the SQL host (192.168.0.20
):
shell> mysqld_safe &
On Windows platforms, assuming that you have installed all MySQL Cluster processes as Windows services using the default service names (see Section 17.2.2.4, “Installing MySQL Cluster Processes as Windows Services”), you can restart the cluster as follows:
On the management host (192.168.0.10
in our
example setup), execute the following command:
C:\> NET START ndb_mgmd
On each of the data node hosts
(192.168.0.30
and
192.168.0.40
), execute the following
command:
C:\> NET START ndbd
On the management node host, use the ndb_mgm client to verify that the management node and both data nodes have started successfully (see Section 17.2.2.3, “Initial Startup of MySQL Cluster on Windows”).
On the SQL node host (192.168.0.20
),
execute the following command:
C:\> NET START mysql
In a production setting, it is usually not desirable to shut down the cluster completely. In many cases, even when making configuration changes, or performing upgrades to the cluster hardware or software (or both), which require shutting down individual host machines, it is possible to do so without shutting down the cluster as a whole by performing a rolling restart of the cluster. For more information about doing this, see Section 17.5.5, “Performing a Rolling Restart of a MySQL Cluster”.
This section provides information about MySQL Cluster software and table file compatibility between MySQL 5.1.23 and earlier, MySQL Cluster NDB 6.x, and MySQL Cluster NDB 7.x releases with regard to performing upgrades and downgrades as well as compatibility matrices and notes. You are expected already to be familiar with installing and configuring a MySQL Cluster prior to attempting an upgrade or downgrade. See Section 17.3, “Configuration of MySQL Cluster NDB 6.1-7.1”.
For information regarding the rolling restart procedure used to perform an online upgrade, see Section 17.5.5, “Performing a Rolling Restart of a MySQL Cluster”.
Only compatibility between MySQL versions with regard to
NDBCLUSTER
is taken into account in
this section, and there are likely other issues to be
considered. As with any other MySQL software upgrade
or downgrade, you are strongly encouraged to review the relevant
portions of the MySQL Manual for the MySQL versions from which
and to which you intend to migrate, before attempting an upgrade
or downgrade of the MySQL Cluster software. See
Section 2.13.1, “Upgrading MySQL”.
For information about upgrades and downgrades to, from, and between different releases of MySQL Cluster NDB 7.0 and MySQL Cluster NDB 7.1, see Section 17.2.7.1, “Upgrade and downgrade compatibility: MySQL Cluster NDB 7.x”.
For information about upgrades and downgrades to, from, and between different releases of MySQL Cluster NDB 6.1, MySQL Cluster NDB 6.2, and MySQL Cluster NDB 6.3, see Section 17.2.7.2, “Upgrade and Downgrade Compatibility: MySQL Cluster NDB 6.x”.
For information about upgrades and downgrades to, from, and between different releases of MySQL Cluster NDB 7.2, see Upgrading and Downgrading MySQL Cluster NDB 7.2.
For information about upgrades and downgrades to, from, and between different editions of MySQL Cluster as found in standard MySQL 5.1.23 and earlier releases, see Section 17.2.7.3, “Upgrade and downgrade compatibility: MySQL 5.1”.
Only compatibility between MySQL versions with regard to
NDBCLUSTER
is taken into account in
this section, and there are likely other issues to be
considered. As with any other MySQL software upgrade
or downgrade, you are strongly encouraged to review the relevant
portions of the MySQL Manual for the MySQL versions from which
and to which you intend to migrate, before attempting an upgrade
or downgrade of the MySQL Cluster software. See
Section 2.13.1, “Upgrading MySQL”.
The table shown here provides information on MySQL Cluster upgrade and downgrade compatibility among different releases of MySQL Cluster NDB 7.0 and 7.1. Additional notes about upgrades and downgrades to, from, or within the MySQL Cluster NDB 7.x release series can be found immediately following the table.
Versions supported. Online upgrades from any MySQL Cluster NDB 7.0 release up to and including MySQL Cluster NDB 7.0.4 (as well as all early releases numbered NDB 6.4.x) to MySQL Cluster NDB 7.0.5 or later are not possible. Upgrades to MySQL Cluster NDB 7.0.6 or later from MySQL Cluster NDB 6.3.8 or a later MySQL Cluster NDB 6.3 release, or from MySQL Cluster NDB 7.0.5 or later, are supported. (Bug #44294)
Upgrading ndbd to ndbmtd. When upgrading online from a MySQL Cluster NDB 6.3 release to a MySQL Cluster NDB 7.0 (or later) release, you should not try to upgrade the data nodes from ndbd to ndbmtd at the same time. Instead, perform the upgrade using the new ndbd executable (from the MySQL Cluster NDB 7.0 or later distribution to which you are upgrading) to replace the one in use on the data nodes. Once the version upgrade is complete, you can perform a second (online) upgrade to replace the data node executables with ndbmtd from the newer distribution. Changing the data node binary type should not be an issue when upgrading from MySQL Cluster NDB 7.0 or later version.
Changes in default values.
In MySQL Cluster NDB 7.0.4, the default values for a number of
MySQL Cluster configuration parameters relating to memory
usage and buffering changed (see
http://dev.mysql.com/doc/relnotes/mysql-cluster/7.1/en/mysql-cluster-news-5-1-32-ndb-7-0-4.html,
for a list of the parameters whose defaults changed). For this
reason, you may encounter issues if you try to use a
configuration that does not explicitly define each of these
buffers (because it was developed for a previous version of
MySQL Cluster),
SendBufferMemory
and
ReceiveBufferMemory
in
particular.
Other known issues include the following:
Prior to MySQL Cluster NDB 7.0.7, DML statements failed if executed while performing an online upgrade from a MySQL Cluster NDB 6.3 release. (Bug #45917)
Following an upgrade from any MySQL Cluster NDB 6.3.x release to MySQL Cluster NDB 7.0.6, DDL and backup operations failed. This issue was resolved in MySQL Cluster NDB 7.0.7. (Bug #46494, Bug #46563)
In some cases, there could be problems with online upgrades from MySQL Cluster NDB 6.3 releases to MySQL Cluster NDB 7.0 releases due to a previous change in the signalling format used between nodes. This issue was corrected in MySQL Cluster NDB 7.0.9.
Once an NDB
table had an
ALTER ONLINE
TABLE
operation performed on it using a MySQL
Cluster NDB 6.3.x release, it could not be upgraded online
to MySQL Cluster NDB 7.0. This issue was resolved in MySQL
Cluster NDB 7.0.8. (See Bug #47542.)
Following an upgrade from MySQL Cluster NDB 6.3 to MySQL
Cluster NDB 7.0, if there were any tables having unique
indexes prior to the upgrade, attempts to create unique
indexes failed. This could also occur when performing
offline ALTER TABLE
operations on tables having indexes that were not dropped as
a result of the ALTER TABLE
.
This issue was due to a change in the way that
NDB
tracked unique indexes
internally, and was resolved in MySQL Cluster NDB 7.0.9.
(Bug #48416)
For upgrades to MySQL Cluster NDB 7.0 releases prior to
version 7.0.9, a workaround is available: Following the
upgrade, execute a second rolling
restart of the cluster before performing any
ALTER TABLE
operations
involving indexes.
Due to an issue discovered after the release of MySQL Cluster NDB 7.0.10 (Bug #50433), it is not possible to perform an online upgrade from MySQL Cluster NDB 7.0.9b and earlier MySQL Cluster NDB 7.0 releases to MySQL Cluster NDB 7.0.10. Instead, you should upgrade your MySQL Cluster NDB 7.0 installation directly to MySQL Cluster NDB 7.0.11 or later.
This issue did not appear to affect MySQL Cluster NDB 6.3, and it should be possible to upgrade online from MySQL Cluster NDB 6.3 to MySQL Cluster NDB 7.0.10 without any problems other than those noted previously.
It was not possible to perform an online upgrade from a MySQL Cluster NDB 6.3 or 7.0 release to MySQL Cluster NDB 7.1.0 or 7.1.1. This issue was fixed in MySQL Cluster NDB 7.1.2 (see Bug #51429).
Following an upgrade to MySQL Cluster NDB 7.0.15 or later
(MySQL Cluster NDB 7.0), or to MySQL Cluster NDB 7.1.4 or
later (MySQL Cluster NDB 7.1), a table created in a previous
version of MySQL Cluster does not
automatically support
NDB
-native default values. Such
a table continues to use default values supplied by the
MySQL server until it is upgraded by performing an offline
ALTER TABLE
on it.
When upgrading to a MySQL Cluster release that supports native default values from a MySQL Cluster release that does not, you should not attempt to create any new tables until all data nodes are using the new ndbd or ndbmtd binaries. This is because the older binaries do not provide support for native default values.
Tables created with native default value support cannot be used with versions of MySQL Cluster that do not support native default values.
NDB API changes in MySQL Cluster NDB 7.1.16 and MySQL Cluster NDB 7.0.27 are not backward compatible. Due to these changes the version of the included NDB client library was increased from 4.0.0 to 5.0.0; NDB API applications must be recompiled as part of any upgrade to these or later MySQL Cluster releases.
In addition, NDB API programs should be updated as part of
such an upgrade, in order to take into account changes in
the default behavior and values for the
Ndb_cluster_connection::connect()
method.
Due to issues discovered shortly after release, MySQL Cluster NDB 7.0.20 was withdrawn and replaced with MySQL Cluster NDB 7.0.20a. Users of MySQL Cluster NDB 7.0.19 and previous MySQL Cluster NDB 7.0 releases should upgrade to MySQL Cluster NDB 7.0.20a, or to a later MySQL Cluster NDB 7.0 release. See http://dev.mysql.com/doc/relnotes/mysql-cluster/7.1/en/mysql-cluster-news-5-1-51-ndb-7-0-20a.html, for more information.
Due to issues discovered following release, MySQL Cluster NDB 7.1.15 was withdrawn and replaced with MySQL Cluster NDB 7.1.15a. Users of MySQL Cluster NDB 7.1.14 and previous MySQL Cluster NDB 7.1 releases should upgrade to MySQL Cluster NDB 7.1.15a, or to a later MySQL Cluster NDB 7.1 release. See http://dev.mysql.com/doc/relnotes/mysql-cluster/7.1/en/mysql-cluster-news-5-1-56-ndb-7-1-15a.html, for more information.
Due to issues discovered shortly after release, MySQL Cluster NDB 7.1.9 was withdrawn and replaced with MySQL Cluster NDB 7.1.9a. Users of MySQL Cluster NDB 7.1.8 and previous MySQL Cluster NDB 7.1 releases should upgrade to MySQL Cluster NDB 7.1.14 (the last release directly upgrade-compatible with MySQL Cluster 7.1.18 and earlier) before upgrading to MySQL Cluster 7.1.15a or later. See http://dev.mysql.com/doc/relnotes/mysql-cluster/7.1/en/mysql-cluster-news-5-1-51-ndb-7-1-9a.html, for more information.
When performing an online upgrade or downgrade between MySQL
Cluster NDB 7.1.8 or earlier and a later release up to and
including MySQL Cluster NDB 7.1.14, you must upgrade or
downgrade the data nodes before upgrading or downgrading any
SQL nodes; otherwise mysql_upgrade fails
on the SQL nodes due to differences between
ndbinfo
tables used in the
“old” and “new” versions of the
MySQL Cluster software. You should also upgrade or downgrade
the data nodes prior to the SQL nodes when performing an
online upgrade or downgrade between MySQL Cluster NDB 7.1
releases where either of the versions involved is MySQL
Cluster NDB 7.1.14 or earlier, and where one or more
ndbinfo
tables has more,
fewer, or differing columns between the two versions.
This issue is resolved in MySQL Cluster NDB 7.1.15. (Bug #11885602)
It was not possible to downgrade online to MySQL Cluster NDB
7.1 from some later versions of MySQL Cluster due to a
change in the size used for table hash maps by
NDB
. (Bug #14645319) This issue
was resolved in MySQL Cluster NDB 7.1.26, where the size is
made configurable using the
DefaultHashMapSize
parameter. (Bug #14800539) See the description of this
parameter for more information.
The table shown here provides information on MySQL Cluster upgrade and downgrade compatibility among different releases of MySQL Cluster NDB 6.1, 6.2, and 6.3. Additional notes about upgrades and downgrades to, from, or within the MySQL Cluster NDB 6.x release series can be found immediately following the table.
Availability of older releases. MySQL Cluster NDB 6.1 is no longer in production; information about this series of releases is of historical interest only. MySQL Cluster NDB 6.2 is still available, but is no longer supported in new deployments. If you are still using a MySQL Cluster NDB 6.1 or MySQL Cluster NDB 6.2 release, you should upgrade to the most recent MySQL Cluster NDB 7.0 or MySQL Cluster NDB 7.1 release as soon as possible.
It is not possible to upgrade from MySQL Cluster NDB 6.1.2 (or an older 6.1 release) directly to 6.1.4 or a newer NDB 6.1 release, or to downgrade from 6.1.4 (or a newer 6.1 release) directly to 6.1.2 or an older NDB 6.1 release; in either case, you must upgrade or downgrade to MySQL Cluster NDB 6.1.3 first.
It is not possible to perform an online downgrade from MySQL Cluster NDB 6.1.8 (or a newer 6.1 release) to MySQL Cluster NDB 6.1.7 (or an older 6.1 release).
MySQL Cluster NDB 6.1.6 and 6.1.18 were not released.
It is not possible to perform an online upgrade or downgrade between MySQL Cluster NDB 6.2 and any previous release series (including mainline MySQL 5.1 and MySQL Cluster NDB 6.1); it is necessary to perform a dump and reload. However, it should be possible to perform online upgrades or downgrades between any MySQL Cluster NDB 6.2 release and any MySQL Cluster NDB 6.3 release up to and including 6.3.7.
Internal column specification changes.
The internal specifications for columns in
NDB
tables changed in MySQL
Cluster NDB 6.1.17 and 6.2.1 to enable compatibility with
later MySQL Cluster releases implementing online adding and
dropping of columns (MySQL 5.1.17 through MySQL 5.1.23; MySQL
Cluster NDB 6.2.3 and later; MySQL Cluster NDB 6.3.2 and
later). This change is not backward-compatible with earlier
MySQL Server or MySQL Cluster releases.
To make tables created in earlier versions compatible with online adding and dropping of columns in later versions, it is necessary to force MySQL Cluster to convert the tables to the new format by following this procedure following an upgrade:
Upgrade the MySQL Cluster software on all data, management, and SQL nodes
Back up all NDB
tables
Shut down the cluster (all data, management, and SQL nodes)
Restart the cluster, starting all data nodes with the
--initial
option (to clear and
rebuild the data node file systems)
Restore the tables from backup
To minimize possible later difficulties, it is strongly advised
that the procedure outlined above be followed as soon as
possible after to upgrading between the versions indicated. The
procedure is not necessary for
NDBCLUSTER
tables created in any of
the following versions:
MySQL Cluster NDB 6.1.8 or a later MySQL Cluster NDB 6.1 release
MySQL Cluster 6.2.1 or a later MySQL Cluster NDB 6.2 release
Any MySQL Cluster NDB 6.3 release (or later MySQL Cluster release series)
Tables created in the versions listed previously (or later versions, as indicated) are already compatible with adding and dropping of columns online (as implemented beginning with MySQL Cluster NDB 6.2.5 and MySQL Cluster NDB 6.3.2).
Additional issues encountered when upgrading or downgrading to or from MySQL Cluster NDB 6.2 and MySQL Cluster NDB 6.3 releases are listed here:
It was not possible to perform an online upgrade between any MySQL Cluster NDB 6.2 release and MySQL Cluster NDB 6.3.8 and later MySQL Cluster 6.3 releases. This issue was fixed in MySQL Cluster NDB 6.3.21. (Bug #41435)
Online downgrades between MySQL Cluster NDB 6.2.5 and earlier releases are not supported.
Online downgrades between MySQL Cluster NDB 6.3.8 and earlier releases are not supported.
The table shown here provides information on MySQL Cluster upgrade and downgrade compatibility among different releases of MySQL 5.1 prior to MySQL 5.1.24. Additional notes about upgrades and downgrades to, from, or within the MySQL 5.1 release series can be found immediately following the table.
MySQL 5.1.3 was the first public release in this series.
Direct upgrades or downgrades between MySQL Cluster 5.0 and
5.1 are not supported; you must dump all
NDBCLUSTER
tables using
mysqldump, install the new version of the
software, and then reload the tables from the dump.
You cannot downgrade a MySQL Cluster based on MySQL 5.1.6 or
later and using Disk Data tables to MySQL 5.1.5 or earlier
unless you convert all such tables to in-memory
NDB
tables first.
MySQL 5.1.8, MySQL 5.1.10, and MySQL 5.1.13 were not released.
Online cluster upgrades and downgrades between MySQL 5.1.11
(or an earlier version) and 5.1.12 (or a later version) are
not possible due to major changes in the cluster file
system. In such cases, you must perform a backup or dump,
upgrade (or downgrade) the software, start each data node
with --initial
, and then
restore from the backup or dump. You can use native
NDB
backup and restore, or
mysqldump and
LOAD DATA
INFILE
for this purpose.
Online downgrades from MySQL 5.1.14 or later to versions previous to 5.1.14 are not supported due to incompatible changes in the cluster system tables.
MySQL Cluster Replication: changes in ndb_apply_status.
Online upgrades from MySQL 5.1.17 and earlier to 5.1.18 and
later MySQL 5.1.x releases are not supported for clusters
using replication due to incompatible changes in the
mysql.ndb_apply_status
table. (Online
upgrades from MySQL 5.1 to MySQL Cluster NDB 6.2 and later are
not supported, as discussed elsewhere in this section.)
However, it should not be necessary to shut down the cluster
entirely, if you follow this modified rolling restart
procedure:
Stop the management server, update the management server software, then start the management server again. For multiple management servers, repeat this step for each management server in turn.
For each data node in turn: Stop the data node, update the
data node daemon (in MySQL Cluster NDB 7.0 and later, this
can be either ndbd or
ndbmtd) with the new version, then
restart the data node. It should not be necessary to use
--initial
when restarting any of the data
nodes after updating the software.
Stop all SQL nodes. Upgrade the
existing MySQL server installations to the new version on
all SQL nodes, then restart them. It is not necessary to
start them one at a time after upgrading the MySQL server
software, but there must be a time when none of
them is running before starting any of them again using the
5.1.18 (or later) mysqld.
Otherwise—due to the fact that
mysql.ndb_apply_status
uses the
NDB
storage engine and is thus
shared between all SQL nodes—there may be conflicts
between the old and new versions of the table on different
SQL nodes.
You can find more information about the changes to
ndb_apply_status
in
Section 17.6.4, “MySQL Cluster Replication Schema and Tables”.
You should upgrade the MySQL Cluster software on each node using the same method by which it was originally installed. See Section 17.2, “MySQL Cluster Installation and Upgrades”, for more information.
As with any other MySQL Cluster version upgrade, you should also update the MySQL Cluster management client (ndb_mgm) and other MySQL Cluster client programs such as ndb_config and ndb_error_reporter; however, this does not have to be done in any particular order.
Internal column specification changes.
The internal specifications for columns in
NDBCLUSTER
tables changed in
MySQL 5.1.18 to enable compatibility with later MySQL Cluster
releases that permit online adding and dropping of columns.
This change is not backward-compatible with earlier
MySQL versions.
To make tables created in MySQL 5.1.17 and earlier compatible with online adding and dropping of columns (available beginning with beginning with MySQL Cluster NDB 6.2.5 and MySQL Cluster NDB 6.3.2—see Section 13.1.7, “ALTER TABLE Syntax”, for more information), it is necessary to force MySQL 5.1.18 and later to convert the tables to the new format by following this procedure:
Back up all NDBCLUSTER
tables.
Upgrade the MySQL Cluster software on all data, management, and SQL nodes.
Shut down the cluster completely (this includes all data, management, and API or SQL nodes).
Restart the cluster, starting all data nodes with the
--initial
option (to clear and
rebuild the data node file systems).
Restore the NDBCLUSTER
tables
from backup.
It is not necessary to follow this procedure for
NDBCLUSTER
tables created in MySQL
5.1.18 and later; such tables are already compatible with online
adding and dropping of columns (as implemented beginning with
MySQL Cluster NDB 6.2.5 and MySQL Cluster NDB 6.3.2).
To minimize possible later difficulties, it is strongly advised that the procedure outlined above be followed as soon as possible after to upgrading from MySQL 5.1.17 or earlier to MySQL 5.1.18 or later.
Information about how this change effects users of MySQL Cluster NDB 6.x and 7.x is provided later in this section.
MySQL Cluster not supported in MySQL Server 5.1.24 and later. MySQL Cluster is not supported in standard MySQL 5.1 releases beginning with MySQL 5.1.24. If you are using MySQL Cluster in a standard MySQL 5.1 release, you should upgrade to the most recent MySQL Cluster NDB 7.0 or MySQL Cluster NDB 7.1 release.
A MySQL server that is part of a MySQL Cluster differs in one chief
respect from a normal (nonclustered) MySQL server, in that it
employs the NDB
storage engine. This
engine is also referred to sometimes as
NDBCLUSTER
, although
NDB
is preferred.
To avoid unnecessary allocation of resources, the server is
configured by default with the NDB
storage engine disabled. To enable NDB
,
you must modify the server's my.cnf
configuration file, or start the server with the
--ndbcluster
option.
This MySQL server is a part of the cluster, so it also must know how
to access an management node to obtain the cluster configuration
data. The default behavior is to look for the management node on
localhost
. However, should you need to specify
that its location is elsewhere, this can be done in
my.cnf
, or with the mysql
client. Before the NDB
storage engine
can be used, at least one management node must be operational, as
well as any desired data nodes.
For more information about
--ndbcluster
and other
mysqld options specific to MySQL Cluster, see
Section 17.3.4.2, “MySQL Server Options for MySQL Cluster”.
For information about installing MySQL Cluster, see Section 17.2, “MySQL Cluster Installation and Upgrades”.
To familiarize you with the basics, we will describe the simplest possible configuration for a functional MySQL Cluster. After this, you should be able to design your desired setup from the information provided in the other relevant sections of this chapter.
First, you need to create a configuration directory such as
/var/lib/mysql-cluster
, by executing the
following command as the system root
user:
shell> mkdir /var/lib/mysql-cluster
In this directory, create a file named
config.ini
that contains the following
information. Substitute appropriate values for
HostName
and DataDir
as
necessary for your system.
# file "config.ini" - showing minimal setup consisting of 1 data node,
# 1 management server, and 3 SQL nodes.
# The empty default sections are not required, and are shown only for
# the sake of completeness.
# Data nodes must provide a hostname but SQL nodes are not required
# to do so.
# If you do not know the hostname for your machine, use localhost.
# The DataDir parameter also has a default value, but it is recommended to
# set it explicitly.
# Note: [db], [api], and [mgm] are aliases for [ndbd], [mysqld], and [ndb_mgmd],
# respectively. [db] is deprecated and should not be used in new installations.
[ndbd default]
NoOfReplicas= 1
[mysqld default]
[ndb_mgmd default]
[tcp default]
[ndb_mgmd]
HostName= myhost.example.com
[ndbd]
HostName= myhost.example.com
DataDir= /var/lib/mysql-cluster
[mysqld]
[mysqld]
[mysqld]
You can now start the ndb_mgmd management
server. By default, it attempts to read the
config.ini
file in its current working
directory, so change location into the directory where the file is
located and then invoke ndb_mgmd:
shell>cd /var/lib/mysql-cluster
shell>ndb_mgmd
Then start a single data node by running ndbd:
shell> ndbd
For command-line options which can be used when starting ndbd, see Section 17.4.26, “Options Common to MySQL Cluster Programs — Options Common to MySQL Cluster Programs”.
By default, ndbd looks for the management
server at localhost
on port 1186.
If you have installed MySQL from a binary tarball, you will need
to specify the path of the ndb_mgmd and
ndbd servers explicitly. (Normally, these
will be found in /usr/local/mysql/bin
.)
Finally, change location to the MySQL data directory (usually
/var/lib/mysql
or
/usr/local/mysql/data
), and make sure that
the my.cnf
file contains the option necessary
to enable the NDB storage engine:
[mysqld] ndbcluster
You can now start the MySQL server as usual:
shell> mysqld_safe --user=mysql &
Wait a moment to make sure the MySQL server is running properly.
If you see the notice mysql ended
, check the
server's .err
file to find out what went
wrong.
If all has gone well so far, you now can start using the cluster.
Connect to the server and verify that the
NDB
storage engine is enabled:
shell>mysql
Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 to server version: 5.1.73 Type 'help;' or '\h' for help. Type '\c' to clear the buffer. mysql>SHOW ENGINES\G
... *************************** 12. row *************************** Engine: NDBCLUSTER Support: YES Comment: Clustered, fault-tolerant, memory-based tables *************************** 13. row *************************** Engine: NDB Support: YES Comment: Alias for NDBCLUSTER ...
The row numbers shown in the preceding example output may be different from those shown on your system, depending upon how your server is configured.
Try to create an NDB
table:
shell>mysql
mysql>USE test;
Database changed mysql>CREATE TABLE ctest (i INT) ENGINE=NDBCLUSTER;
Query OK, 0 rows affected (0.09 sec) mysql>SHOW CREATE TABLE ctest \G
*************************** 1. row *************************** Table: ctest Create Table: CREATE TABLE `ctest` ( `i` int(11) default NULL ) ENGINE=ndbcluster DEFAULT CHARSET=latin1 1 row in set (0.00 sec)
To check that your nodes were set up properly, start the management client:
shell> ndb_mgm
Use the SHOW command from within the management client to obtain a report on the cluster's status:
ndb_mgm> SHOW
Cluster Configuration
---------------------
[ndbd(NDB)] 1 node(s)
id=2 @127.0.0.1 (Version: 5.1.73-ndb-7.1.34, Nodegroup: 0, *)
[ndb_mgmd(MGM)] 1 node(s)
id=1 @127.0.0.1 (Version: 5.1.73-ndb-7.1.34)
[mysqld(API)] 3 node(s)
id=3 @127.0.0.1 (Version: 5.1.73-ndb-7.1.34)
id=4 (not connected, accepting connect from any host)
id=5 (not connected, accepting connect from any host)
At this point, you have successfully set up a working MySQL
Cluster. You can now store data in the cluster by using any table
created with ENGINE=NDBCLUSTER
or its alias
ENGINE=NDB
.
Configuring MySQL Cluster requires working with two files:
my.cnf
: Specifies options for all MySQL
Cluster executables. This file, with which you should be
familiar with from previous work with MySQL, must be
accessible by each executable running in the cluster.
config.ini
: This file, sometimes known as
the global configuration file, is read
only by the MySQL Cluster management server, which then
distributes the information contained therein to all processes
participating in the cluster. config.ini
contains a description of each node involved in the cluster.
This includes configuration parameters for data nodes and
configuration parameters for connections between all nodes in
the cluster. For a quick reference to the sections that can
appear in this file, and what sorts of configuration
parameters may be placed in each section, see
Sections of
the config.ini
File.
Caching of configuration data. Beginning with MySQL Cluster NDB 6.4.0, MySQL Cluster uses stateful configuration. The global configuration file is no longer read every time the management server is restarted. Instead, the management server caches the configuration the first time it is started, and thereafter, the global configuration file is read only when one of the following items is true:
The management server is started using the --initial option. In this case, the global configuration file is re-read, any existing cache files are deleted, and the management server creates a new configuration cache.
The management server is started using the --reload option. In this case, the management server compares its cache with the global configuration file. If they differ, the management server creates a new configuration cache; any existing configuration cache is preserved, but not used. If the management server's cache and the global configuration file contain the same configuration data, then the existing cache is used, and no new cache is created.
The management server is started using a --config-cache option.
Beginning with MySQL Cluster NDB 7.0.15 and MySQL Cluster
NDB 7.1.4, this option can be used to force the management
server to bypass configuration caching altogether. In this
case, the management server ignores any configuration files
that may be present, always reading its configuration data
from the config.ini
file instead.
No configuration cache is found. In this case, the management server reads the global configuration file and creates a cache containing the same configuration data as found in the file.
Configuration cache files.
Beginning with MySQL Cluster 6.4.0, the management server by
default creates configuration cache files in a directory named
mysql-cluster
in the MySQL installation
directory. (If you build MySQL Cluster from source on a Unix
system, the default location is
/usr/local/mysql-cluster
.) This can be
overridden at runtime by starting the management server with the
--configdir
option.
Configuration cache files are binary files named according to
the pattern
ndb_
,
where node_id
_config.bin.seq_id
node_id
is the management
server's node ID in the cluster, and
seq_id
is a cache idenitifer. Cache
files are numbered sequentially using
seq_id
, in the order in which they
are created. The management server uses the latest cache file as
determined by the seq_id
.
It is possible to roll back to a previous configuration by
deleting later configuration cache files, or by renaming an
earlier cache file so that it has a higher
seq_id
. However, since configuration
cache files are written in a binary format, you should not
attempt to edit their contents by hand.
For more information about the
--configdir
,
--config-cache
,
--initial
, and
--reload
options for the MySQL
Cluster management server, see
Section 17.4.4, “ndb_mgmd — The MySQL Cluster Management Server Daemon”.
We are continuously making improvements in Cluster configuration and attempting to simplify this process. Although we strive to maintain backward compatibility, there may be times when introduce an incompatible change. In such cases we will try to let Cluster users know in advance if a change is not backward compatible. If you find such a change and we have not documented it, please report it in the MySQL bugs database using the instructions given in Section 1.7, “How to Report Bugs or Problems”.
To support MySQL Cluster, you will need to update
my.cnf
as shown in the following example.
You may also specify these parameters on the command line when
invoking the executables.
The options shown here should not be confused with those that
are used in config.ini
global
configuration files. Global configuration options are
discussed later in this section.
# my.cnf # example additions to my.cnf for MySQL Cluster # (valid in MySQL 5.1) # enable ndbcluster storage engine, and provide connection string for # management server host (default port is 1186) [mysqld] ndbcluster ndb-connectstring=ndb_mgmd.mysql.com # provide connection string for management server host (default port: 1186) [ndbd] connect-string=ndb_mgmd.mysql.com # provide connection string for management server host (default port: 1186) [ndb_mgm] connect-string=ndb_mgmd.mysql.com # provide location of cluster configuration file [ndb_mgmd] config-file=/etc/config.ini
(For more information on connection strings, see Section 17.3.2.3, “MySQL Cluster Connection Strings”.)
# my.cnf # example additions to my.cnf for MySQL Cluster # (will work on all versions) # enable ndbcluster storage engine, and provide connection string for management # server host to the default port 1186 [mysqld] ndbcluster ndb-connectstring=ndb_mgmd.mysql.com:1186
Once you have started a mysqld process with
the NDBCLUSTER
and
ndb-connectstring
parameters in the
[mysqld]
in the my.cnf
file as shown previously, you cannot execute any
CREATE TABLE
or
ALTER TABLE
statements without
having actually started the cluster. Otherwise, these
statements will fail with an error. This is by
design.
You may also use a separate [mysql_cluster]
section in the cluster my.cnf
file for
settings to be read and used by all executables:
# cluster-specific settings [mysql_cluster] ndb-connectstring=ndb_mgmd.mysql.com:1186
For additional NDB
variables that
can be set in the my.cnf
file, see
Section 17.3.4.3, “MySQL Cluster System Variables”.
The MySQL Cluster global configuration file is by convention
named config.ini
(but this is not
required). It is read by ndb_mgmd at startup
and can be placed in any location that can be read by it. The
location and name of the configuration are specified using
--config-file=
with ndb_mgmd on the command line. Beginning
with MySQL Cluster NDB 6.4.0, this option has no default value,
and is ignored if ndb_mgmd uses the
configuration cache.
path_name
The global configuration file for MySQL Cluster uses INI format,
which consists of sections preceded by section headings
(surrounded by square brackets), followed by the appropriate
parameter names and values. One deviation from the standard INI
format is that the parameter name and value can be separated by
a colon (“:
”) as well as the
equal sign (“=
”); however, the
equal sign is preferred. Another deviation is that sections are
not uniquely identified by section name. Instead, unique
sections (such as two different nodes of the same type) are
identified by a unique ID specified as a parameter within the
section.
Default values are defined for most parameters, and can also be
specified in config.ini
. (Prior to MySQL
Cluster NDB 6.3.25 and MySQL Cluster NDB 7.0.6, there was no
default value for
NoOfReplicas
, which
always had to be specified explicitly in the [ndbd
default]
section. Beginning with versions just stated,
the default value is 2, which is the recommended setting in most
common usage scenarios.) To create a default value section,
simply add the word default
to the section
name. For example, an [ndbd]
section contains
parameters that apply to a particular data node, whereas an
[ndbd default]
section contains parameters
that apply to all data nodes. Suppose that all data nodes should
use the same data memory size. To configure them all, create an
[ndbd default]
section that contains a
DataMemory
line to
specify the data memory size.
The global configuration file must define the computers and nodes involved in the cluster and on which computers these nodes are located. An example of a simple configuration file for a cluster consisting of one management server, two data nodes and two SQL nodes is shown here:
# file "config.ini" - 2 data nodes and 2 SQL nodes # This file is placed in the startup directory of ndb_mgmd (the # management server) # The first SQL node can be started from any host. The second # can be started only on the host mysqld_5.mysql.com [ndbd default] NoOfReplicas= 2 DataDir= /var/lib/mysql-cluster [ndb_mgmd] Hostname= ndb_mgmd.mysql.com DataDir= /var/lib/mysql-cluster [ndbd] HostName= ndbd_2.mysql.com [ndbd] HostName= ndbd_3.mysql.com [mysqld] [mysqld] HostName= mysqld_5.mysql.com
The preceding example is intended as a minimal starting configuration for purposes of familiarization with MySQL Cluster, and is almost certain not to be sufficient for production settings. See Section 17.3.2.2, “Recommended Starting Configurations for MySQL Cluster NDB 6.2 and Later”, which provides more complete example starting configurations for use with MySQL Cluster NDB 6.2 and newer versions of MySQL Cluster.
Each node has its own section in the
config.ini
file. For example, this cluster
has two data nodes, so the preceding configuration file contains
two [ndbd]
sections defining these nodes.
Do not place comments on the same line as a section heading in
the config.ini
file; this causes the
management server not to start because it cannot parse the
configuration file in such cases.
There are six different sections that you can use in the
config.ini
configuration file, as described
in the following list:
[computer]
: Defines cluster hosts. This
is not required to configure a viable MySQL Cluster, but be
may used as a convenience when setting up a large cluster.
See Section 17.3.2.4, “Defining Computers in a MySQL Cluster”, for
more information.
[ndbd]
: Defines a cluster data node
(ndbd process). See
Section 17.3.2.6, “Defining MySQL Cluster Data Nodes”, for
details.
[mysqld]
: Defines the cluster's MySQL
server nodes (also called SQL or API nodes). For a
discussion of SQL node configuration, see
Section 17.3.2.7, “Defining SQL and Other API Nodes in a MySQL Cluster”.
[mgm]
or [ndb_mgmd]
:
Defines a cluster management server (MGM) node. For
information concerning the configuration of management
nodes, see Section 17.3.2.5, “Defining a MySQL Cluster Management Server”.
[tcp]
: Defines a TCP/IP connection
between cluster nodes, with TCP/IP being the default
connection protocol. Normally, [tcp]
or
[tcp default]
sections are not required
to set up a MySQL Cluster, as the cluster handles this
automatically; however, it may be necessary in some
situations to override the defaults provided by the cluster.
See Section 17.3.2.8, “MySQL Cluster TCP/IP Connections”, for
information about available TCP/IP configuration parameters
and how to use them. (You may also find
Section 17.3.2.9, “MySQL Cluster TCP/IP Connections Using Direct Connections” to be
of interest in some cases.)
[shm]
: Defines shared-memory connections
between nodes. In MySQL 5.1, it is enabled by
default, but should still be considered experimental. For a
discussion of SHM interconnects, see
Section 17.3.2.10, “MySQL Cluster Shared-Memory Connections”.
[sci]
:Defines Scalable
Coherent Interface connections between cluster
data nodes. Such connections require software which, while
freely available, is not part of the MySQL Cluster
distribution, as well as specialized hardware. See
Section 17.3.2.11, “SCI Transport Connections in MySQL Cluster” for detailed
information about SCI interconnects.
You can define default
values for each
section. All Cluster parameter names are case-insensitive, which
differs from parameters specified in my.cnf
or my.ini
files.
Achieving the best performance from a MySQL Cluster depends on a number of factors including the following:
MySQL Cluster software version
Numbers of data nodes and SQL nodes
Hardware
Operating system
Amount of data to be stored
Size and type of load under which the cluster is to operate
Therefore, obtaining an optimum configuration is likely to be an iterative process, the outcome of which can vary widely with the specifics of each MySQL Cluster deployment. Changes in configuration are also likely to be indicated when changes are made in the platform on which the cluster is run, or in applications that use the MySQL Cluster's data. For these reasons, it is not possible to offer a single configuration that is ideal for all usage scenarios. However, in this section, we provide recommended base configurations for MySQL Cluster NDB 6.2 and 6.3 that can serve as reasonable starting points.
Starting configuration for MySQL Cluster NDB 6.2. The following is a recommended starting point for configuring a cluster running MySQL Cluster NDB 6.2.
# TCP PARAMETERS [tcp default]SendBufferMemory
=2MReceiveBufferMemory
=2M # Increasing the sizes of these 2 buffers beyond the default values # helps prevent bottlenecks due to slow disk I/O. # MANAGEMENT NODE PARAMETERS [ndb_mgmd default]DataDir
=path/to/management/server/data/directory
# It is possible to use a different data directory for each management # server, but for ease of administration it is preferable to be # consistent. [ndb_mgmd]HostName
=management-server-1-hostname
#NodeId
=management-server-1-nodeid
[ndb_mgmd]HostName
=management-server-2-hostname
# Using 2 management servers helps guarantee that there is always an # arbitrator in the event of network partitioning, and so is # recommended for high availability. Each management server must be # identified by a HostName. You may for the sake of convenience specify # a node ID for any management server, although one will be allocated # for it automatically; if you do so, it must be in the range 1-255 # inclusive and must be unique among all IDs specified for cluster # nodes. # DATA NODE PARAMETERS [ndbd default]NoOfReplicas
=2 # Using 2 replicas is recommended to guarantee availability of data; # using only 1 replica does not provide any redundancy, which means # that the failure of a single data node causes the entire cluster to # shut down. We do not recommend using more than 2 replicas, since 2 is # sufficient to provide high availability, and we do not currently test # with greater values for this parameter.LockPagesInMainMemory
=1 # On Linux and Solaris systems, setting this parameter locks data node # processes into memory. Doing so prevents them from swapping to disk, # which can severely degrade cluster performance.DataMemory
=3072MIndexMemory
=384M # The values provided for DataMemory and IndexMemory assume 4 GB RAM # per data node. However, for best results, you should first calculate # the memory that would be used based on the data you actually plan to # store (you may find the ndb_size.pl utility helpful in estimating # this), then allow an extra 20% over the calculated values. Naturally, # you should ensure that each data node host has at least as much # physical memory as the sum of these two values. #ODirect
=1 # Enabling this parameter causes NDBCLUSTER to try using O_DIRECT # writes for local checkpoints and redo logs; this can reduce load on # CPUs. We recommend doing so when using MySQL Cluster NDB 6.2.3 or # newer on systems running Linux kernel 2.6 or later.NoOfFragmentLogFiles
=300DataDir
=path/to/data/node/data/directory
MaxNoOfConcurrentOperations
=100000TimeBetweenGlobalCheckpoints
=1000TimeBetweenEpochs
=200DiskCheckpointSpeed
=10MDiskCheckpointSpeedInRestart
=100MRedoBuffer
=32M #MaxNoOfLocalScans
=64MaxNoOfTables
=1024MaxNoOfOrderedIndexes
=256 [ndbd]HostName
=data-node-A-hostname
#NodeId
=data-node-A-nodeid
[ndbd]HostName
=data-node-B-hostname
#NodeId
=data-node-B-nodeid
# You must have an [ndbd] section for every data node in the cluster; # each of these sections must include a HostName. Each section may # optionally include an Id for convenience, but in most cases, it is # sufficient to allow the cluster to allocate node IDs dynamically. If # you do specify the node ID for a data node, it must be in the range 1 # to 48 inclusive and must be unique among all IDs specified for # cluster nodes. # SQL NODE / API NODE PARAMETERS [mysqld] #HostName
=SQL-node-1-hostname
#NodeId
=sql-node-A-nodeid
[mysqld] [mysqld] # Each API or SQL node that connects to the cluster requires a [mysqld] # or [api] section of its own. Each such section defines a connection # “slot”; you should have at least as many of these sections in the # config.ini file as the total number of API nodes and SQL nodes that # you wish to have connected to the cluster at any given time. There is # no performance or other penalty for having extra slots available in # case you find later that you want or need more API or SQL nodes to # connect to the cluster at the same time. # If no HostName is specified for a given [mysqld] or [api] section, # then any API or SQL node may use that slot to connect to the # cluster. You may wish to use an explicit HostName for one connection slot # to guarantee that an API or SQL node from that host can always # connect to the cluster. If you wish to prevent API or SQL nodes from # connecting from other than a desired host or hosts, then use a # HostName for every [mysqld] or [api] section in the config.ini file. # You can if you wish define a node ID (Id parameter) for any API or # SQL node, but this is not necessary; if you do so, it must be in the # range 1 to 255 inclusive and must be unique among all IDs specified # for cluster nodes.
Starting configuration for MySQL Cluster NDB 6.3.
The following is a recommended starting point for configuring
a cluster running MySQL Cluster NDB 6.3. It is similar to the
recommendation for MySQL Cluster NDB 6.2, with the addition of
parameters for better control of
NDBCLUSTER
process threads.
# TCP PARAMETERS [tcp default]SendBufferMemory
=2MReceiveBufferMemory
=2M # Increasing the sizes of these 2 buffers beyond the default values # helps prevent bottlenecks due to slow disk I/O. # MANAGEMENT NODE PARAMETERS [ndb_mgmd default]DataDir
=path/to/management/server/data/directory
# It is possible to use a different data directory for each management # server, but for ease of administration it is preferable to be # consistent. [ndb_mgmd]HostName
=management-server-A-hostname
#NodeId
=management-server-A-nodeid
[ndb_mgmd]HostName
=management-server-B-hostname
#NodeId
=management-server-B-nodeid
# Using 2 management servers helps guarantee that there is always an # arbitrator in the event of network partitioning, and so is # recommended for high availability. Each management server must be # identified by a HostName. You may for the sake of convenience specify # a NodeId for any management server, although one will be allocated # for it automatically; if you do so, it must be in the range 1-255 # inclusive and must be unique among all IDs specified for cluster # nodes. # DATA NODE PARAMETERS [ndbd default]NoOfReplicas
=2 # Using 2 replicas is recommended to guarantee availability of data; # using only 1 replica does not provide any redundancy, which means # that the failure of a single data node causes the entire cluster to # shut down. We do not recommend using more than 2 replicas, since 2 is # sufficient to provide high availability, and we do not currently test # with greater values for this parameter.LockPagesInMainMemory
=1 # On Linux and Solaris systems, setting this parameter locks data node # processes into memory. Doing so prevents them from swapping to disk, # which can severely degrade cluster performance.DataMemory
=3072MIndexMemory
=384M # The values provided for DataMemory and IndexMemory assume 4 GB RAM # per data node. However, for best results, you should first calculate # the memory that would be used based on the data you actually plan to # store (you may find the ndb_size.pl utility helpful in estimating # this), then allow an extra 20% over the calculated values. Naturally, # you should ensure that each data node host has at least as much # physical memory as the sum of these two values. #ODirect
=1 # Enabling this parameter causes NDBCLUSTER to try using O_DIRECT # writes for local checkpoints and redo logs; this can reduce load on # CPUs. We recommend doing so when using MySQL Cluster NDB 6.2.3 or # newer on systems running Linux kernel 2.6 or later.NoOfFragmentLogFiles
=300DataDir
=path/to/data/node/data/directory
MaxNoOfConcurrentOperations
=100000SchedulerSpinTimer
=400SchedulerExecutionTimer
=100RealTimeScheduler
=1 # Setting these parameters allows you to take advantage of real-time scheduling # of NDB threads (introduced in MySQL Cluster NDB 6.3.4) to achieve increased # throughput when using ndbd. They are not needed when using ndbmtd; in # particular, you should not setRealTimeScheduler
for ndbmtd data nodes.TimeBetweenGlobalCheckpoints
=1000TimeBetweenEpochs
=200DiskCheckpointSpeed
=10MDiskCheckpointSpeedInRestart
=100MRedoBuffer
=32M #CompressedLCP
=1 #CompressedBackup
=1 # Enabling CompressedLCP and CompressedBackup causes, respectively, local checkpoint files and backup files to be compressed, which can result in a space savings of up to 50% over noncompressed LCPs and backups. #MaxNoOfLocalScans
=64MaxNoOfTables
=1024MaxNoOfOrderedIndexes
=256 [ndbd]HostName
=data-node-A-hostname
#NodeId
=data-node-A-nodeid
LockExecuteThreadToCPU
=1LockMaintThreadsToCPU
=0 # On systems with multiple CPUs, these parameters can be used to lock NDBCLUSTER # threads to specific CPUs [ndbd]HostName
=data-node-B-hostname
#NodeId
=data-node-B-nodeid
LockExecuteThreadToCPU
=1LockMaintThreadsToCPU
=0 # You must have an [ndbd] section for every data node in the cluster; # each of these sections must include a HostName. Each section may # optionally include a NodeId for convenience, but in most cases, it is # sufficient to allow the cluster to allocate node IDs dynamically. If # you do specify the node ID for a data node, it must be in the range 1 # to 48 inclusive and must be unique among all IDs specified for # cluster nodes. # SQL NODE / API NODE PARAMETERS [mysqld] #HostName
=sql-node-A-hostname
#NodeId
=sql-node-A-nodeid
[mysqld] [mysqld] # Each API or SQL node that connects to the cluster requires a [mysqld] # or [api] section of its own. Each such section defines a connection # “slot”; you should have at least as many of these sections in the # config.ini file as the total number of API nodes and SQL nodes that # you wish to have connected to the cluster at any given time. There is # no performance or other penalty for having extra slots available in # case you find later that you want or need more API or SQL nodes to # connect to the cluster at the same time. # If no HostName is specified for a given [mysqld] or [api] section, # then any API or SQL node may use that slot to connect to the # cluster. You may wish to use an explicit HostName for one connection slot # to guarantee that an API or SQL node from that host can always # connect to the cluster. If you wish to prevent API or SQL nodes from # connecting from other than a desired host or hosts, then use a # HostName for every [mysqld] or [api] section in the config.ini file. # You can if you wish define a node ID (NodeId parameter) for any API or # SQL node, but this is not necessary; if you do so, it must be in the # range 1 to 255 inclusive and must be unique among all IDs specified # for cluster nodes.
Recommended my.cnf options for SQL nodes.
MySQL Servers acting as MySQL Cluster SQL nodes must always be
started with the --ndbcluster
and --ndb-connectstring
options, either on
the command line or in my.cnf
. In
addition, set the following options for all
mysqld processes in the cluster, unless
your setup requires otherwise:
--ndb-use-exact-count=0
--ndb-index-stat-enable=0
--ndb-force-send=1
--engine-condition-pushdown=1
With the exception of the MySQL Cluster management server (ndb_mgmd), each node that is part of a MySQL Cluster requires a connection string that points to the management server's location. This connection string is used in establishing a connection to the management server as well as in performing other tasks depending on the node's role in the cluster. The syntax for a connection string is as follows:
[nodeid=node_id
, ]host-definition
[,host-definition
[, ...]]host-definition
:host_name
[:port_number
]
node_id
is an integer greater than or equal
to 1 which identifies a node in config.ini
.
host_name
is a string representing a
valid Internet host name or IP address.
port_number
is an integer referring
to a TCP/IP port number.
example 1 (long): "nodeid=2,myhost1:1100,myhost2:1100,192.168.0.3:1200" example 2 (short): "myhost1"
localhost:1186
is used as the default
connection string value if none is provided. If
port_num
is omitted from the
connection string, the default port is 1186. This port should
always be available on the network because it has been assigned
by IANA for this purpose (see
http://www.iana.org/assignments/port-numbers for
details).
By listing multiple host definitions, it is possible to designate several redundant management servers. A MySQL Cluster data or API node attempts to contact successive management servers on each host in the order specified, until a successful connection has been established.
Beginning with MySQL Cluster NDB 6.3.19, it is also possible in a connection string to specify one or more bind addresses to be used by nodes having multiple network interfaces for connecting to management servers. A bind address consists of a hostname or network address and an optional port number. This enhanced syntax for connection strings is shown here:
[nodeid=node_id
, ] [bind-address=host-definition
, ]host-definition
[; bind-address=host-definition
]host-definition
[; bind-address=host-definition
] [, ...]]host-definition
:host_name
[:port_number
]
If a single bind address is used in the connection string
prior to specifying any management hosts,
then this address is used as the default for connecting to any
of them (unless overridden for a given management server; see
later in this section for an example). For example, the
following connection string causes the node to use
192.168.178.242
regardless of the management
server to which it connects:
bind-address=192.168.178.242, poseidon:1186, perch:1186
If a bind address is specified following a management host definition, then it is used only for connecting to that management node. Consider the following connection string:
poseidon:1186;bind-address=localhost, perch:1186;bind-address=192.168.178.242
In this case, the node uses localhost
to
connect to the management server running on the host named
poseidon
and
192.168.178.242
to connect to the management
server running on the host named perch
.
You can specify a default bind address and then override this
default for one or more specific management hosts. In the
following example, localhost
is used for
connecting to the management server running on host
poseidon
; since
192.168.178.242
is specified first (before
any management server definitions), it is the default bind
address and so is used for connecting to the management servers
on hosts perch
and orca
:
bind-address=192.168.178.242,poseidon:1186;bind-address=localhost,perch:1186,orca:2200
There are a number of different ways to specify the connection string:
Each executable has its own command-line option which enables specifying the management server at startup. (See the documentation for the respective executable.)
It is also possible to set the connection string for all
nodes in the cluster at once by placing it in a
[mysql_cluster]
section in the management
server's my.cnf
file.
For backward compatibility, two other options are available, using the same syntax:
Set the NDB_CONNECTSTRING
environment
variable to contain the connection string.
Write the connection string for each executable into a
text file named Ndb.cfg
and place
this file in the executable's startup directory.
However, these are now deprecated and should not be used for new installations.
The recommended method for specifying the connection string is
to set it on the command line or in the
my.cnf
file for each executable.
Previous to MySQL Cluster NDB 7.0.15 and MySQL Cluster NDB 7.1.4, the maximum length of a connection string was 1024 characters.
The [computer]
section has no real
significance other than serving as a way to avoid the need of
defining host names for each node in the system. All parameters
mentioned here are required.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | string | [none] | ... | IS |
This is a unique identifier, used to refer to the host computer elsewhere in the configuration file.
The computer ID is not the same as
the node ID used for a management, API, or data node.
Unlike the case with node IDs, you cannot use
NodeId
in place of
Id
in the [computer]
section of the config.ini
file.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | name or IP address | [none] | ... | N |
This is the computer's hostname or IP address.
The [ndb_mgmd]
section is used to configure
the behavior of the management server. If multiple management
servers are employed, you can specify parameters common to all
of them in an [ndb_mgmd default]
section.
[mgm]
and [mgm default]
are older aliases for these, supported for backward
compatibility.
All parameters in the following list are optional and assume their default values if omitted.
If neither the ExecuteOnComputer
nor the
HostName
parameter is present, the default
value localhost
will be assumed for both.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | unsigned | [none] | 1 - 63 | IS |
NDB 6.1.1 | unsigned | [none] | 1 - 255 | IS |
Each node in the cluster has a unique identity. For a management node, this is represented by an integer value in the range 1 to 63 inclusive (previous to MySQL Cluster NDB 6.1.1), or in the range 1 to 255 inclusive (MySQL Cluster NDB 6.1.1 and later). This ID is used by all internal cluster messages for addressing the node, and so must be unique for each MySQL Cluster node, regardless of the type of node.
Data node IDs must be less than 49, regardless of the MySQL Cluster version used. If you plan to deploy a large number of data nodes, it is a good idea to limit the node IDs for management nodes (and API nodes) to values greater than 48.
The use of the Id
parameter for
identifying management nodes is deprecated in favor of
NodeId
beginning
with MySQL Cluster NDB 6.2.19, MySQL Cluster NDB 6.3.39,
MySQL Cluster NDB 7.0.20, and MySQL Cluster NDB 7.1.9.
Although Id
continues to be supported for
backward compatibility, it now generates a warning.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | unsigned | [none] | 1 - 63 | IS |
NDB 6.1.1 | unsigned | [none] | 1 - 255 | IS |
Each node in the cluster has a unique identity. For a management node, this is represented by an integer value in the range 1 to 63 inclusive (previous to MySQL Cluster NDB 6.1.1), or in the range 1 to 255 inclusive (MySQL Cluster NDB 6.1.1 and later). This ID is used by all internal cluster messages for addressing the node, and so must be unique for each MySQL Cluster node, regardless of the type of node.
Data node IDs must be less than 49, regardless of the MySQL Cluster version used. If you plan to deploy a large number of data nodes, it is a good idea to limit the node IDs for management nodes (and API nodes) to values greater than 48.
NodeId
is the preferred parameter name to
use when identifying management nodes beginning with MySQL
Cluster NDB 6.2.19, MySQL Cluster NDB 6.3.39, MySQL Cluster
NDB 7.0.20, and MySQL Cluster NDB 7.1.9. Although
Id
continues to be
supported for backward compatibility, it is now deprecated
and generates a warning when used.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | name | [none] | ... | S |
This refers to the Id
set for one of the
computers defined in a [computer]
section
of the config.ini
file.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | unsigned | 1186 | 0 - 64K | N |
This is the port number on which the management server listens for configuration requests and management commands.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | name or IP address | [none] | ... | N |
Specifying this parameter defines the hostname of the
computer on which the management node is to reside. To
specify a hostname other than localhost
,
either this parameter or
ExecuteOnComputer
is required.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | {CONSOLE|SYSLOG|FILE} | [see text] | ... | N |
This parameter specifies where to send cluster logging
information. There are three options in this
regard—CONSOLE
,
SYSLOG
, and
FILE
—with FILE
being the default:
CONSOLE
outputs the log to
stdout
:
CONSOLE
SYSLOG
sends the log to a
syslog
facility, possible values
being one of auth
,
authpriv
, cron
,
daemon
, ftp
,
kern
, lpr
,
mail
, news
,
syslog
, user
,
uucp
, local0
,
local1
, local2
,
local3
, local4
,
local5
, local6
, or
local7
.
Not every facility is necessarily supported by every operating system.
SYSLOG:facility=syslog
FILE
pipes the cluster log output to
a regular file on the same machine. The following values
can be specified:
filename
: The name of the log
file.
Prior to MySQL Cluster NDB 7.0.43 and MySQL Cluster
NDB 7.1.23, the log file's default name, used
if FILE
was specified without
also setting filename
, was
logger.log
. Beginning with
MySQL Cluster NDB 7.0.43 and MySQL Cluster NDB
7.1.23, the default log file name used in such cases
is
ndb_
.
nodeid
_cluster.log
maxsize
: The maximum size (in
bytes) to which the file can grow before logging
rolls over to a new file. When this occurs, the old
log file is renamed by appending
.N
to the file name,
where N
is the next
number not yet used with this name.
maxfiles
: The maximum number of
log files.
FILE:filename=cluster.log,maxsize=1000000,maxfiles=6
The default value for the FILE
parameter is
FILE:filename=ndb_
,
where node_id
_cluster.log,maxsize=1000000,maxfiles=6node_id
is the ID of
the node.
It is possible to specify multiple log destinations separated by semicolons as shown here:
CONSOLE;SYSLOG:facility=local0;FILE:filename=/var/log/mgmd
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | 0-2 | 1 | 0 - 2 | N |
This parameter is used to define which nodes can act as
arbitrators. Only management nodes and SQL nodes can be
arbitrators. ArbitrationRank
can take one
of the following values:
0
: The node will never be used as an
arbitrator.
1
: The node has high priority; that
is, it will be preferred as an arbitrator over
low-priority nodes.
2
: Indicates a low-priority node
which be used as an arbitrator only if a node with a
higher priority is not available for that purpose.
Normally, the management server should be configured as an
arbitrator by setting its ArbitrationRank
to 1 (the default for management nodes) and those for all
SQL nodes to 0 (the default for SQL nodes).
Beginning with MySQL 5.1.16 and MySQL Cluster NDB 6.1.3, it
is possible to disable arbitration completely by setting
ArbitrationRank
to 0 on all management
and SQL nodes. In MySQL Cluster NDB 7.0.7 and later
releases, you can also control arbitration by overriding
this parameter; to do this, set the
Arbitration
parameter in the [ndbd default]
section
of the config.ini
global configuration
file.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | milliseconds | 0 | 0 - 4294967039 (0xFFFFFEFF) | N |
An integer value which causes the management server's responses to arbitration requests to be delayed by that number of milliseconds. By default, this value is 0; it is normally not necessary to change it.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | path | . | ... | N |
This specifies the directory where output files from the
management server will be placed. These files include
cluster log files, process output files, and the daemon's
process ID (PID) file. (For log files, this location can be
overridden by setting the FILE
parameter
for LogDestination
as discussed previously in this section.)
The default value for this parameter is the directory in which ndb_mgmd is located.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | unsigned | [none] | 0 - 64K | N |
This parameter specifies the port number used to obtain statistical information from a MySQL Cluster management server. It has no default value.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.1.1 | boolean | false | true, false | N |
Use WAN TCP setting as default. Introduced in MySQL Cluster NDB 6.1.1.
Beginning with MySQL Cluster NDB 6.3.32, MySQL Cluster NDB 7.0.13, and MySQL Cluster NDB 7.1.2, it is possible to use this parameter to set the scheduling policy and priority of heartbeat threads for management and API nodes.
The syntax for setting this parameter is shown here:
HeartbeatThreadPriority =policy
[,priority
]policy
: {FIFO | RR}
When setting this parameter, you must specify a policy. This
is one of FIFO
(first in, first out) or
RR
(round robin). The policy value is
followed optionally by the priority (an integer).
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 7.0.39 | bytes | 0 | 0 - 32G | N |
This parameter specifies the amount of transporter send
buffer memory to allocate in addition to any that has been
set using
TotalSendBufferMemory
,
SendBufferMemory
, or
both.
This parameter was added in MySQL Cluster NDB 7.0.39 and MySQL Cluster NDB 7.1.28. (Bug #14555359)
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.4.0 | bytes | 256K | 0 - 4294967039 (0xFFFFFEFF) | N |
This parameter is available beginning with MySQL Cluster NDB 6.4.0. It is used to determine the total amount of memory to allocate on this node for shared send buffer memory among all configured transporters.
If this parameter is set, its minimum permitted value is
256KB; the maxmimum is 4294967039. For more detailed
information about the behavior and use of
TotalSendBufferMemory
and configuring
send buffer memory parameters in MySQL Cluster NDB 6.4.0 and
later, see
Section 17.3.2.13, “Configuring MySQL Cluster Send Buffer Parameters”.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 7.0.40 | milliseconds | 1500 | 100 - 4294967039 (0xFFFFFEFF) | N |
NDB 7.1.29 | milliseconds | 1500 | 100 - 4294967039 (0xFFFFFEFF) | N |
Specify the interval between heartbeat messages used to determine whether another management node is on contact with this one. The management node waits after 3 of these intervals to declare the connection dead; thus, the default setting of 1500 milliseconds causes the management node to wait for approximately 1600 ms before timing out.
This parameter was added in MySQL Cluster NDB 7.0.40 and MySQL Cluster NDB 7.1.29. (Bug #16426805)
After making changes in a management node's configuration, it is necessary to perform a rolling restart of the cluster for the new configuration to take effect.
To add new management servers to a running MySQL Cluster, it
is also necessary to perform a rolling restart of all cluster
nodes after modifying any existing
config.ini
files. For more information
about issues arising when using multiple management nodes, see
Section 17.1.6.10, “Limitations Relating to Multiple MySQL Cluster Nodes”.
The [ndbd]
and [ndbd
default]
sections are used to configure the behavior
of the cluster's data nodes.
[ndbd]
and [ndbd default]
are always used as the section names whether you are using
ndbd or (in MySQL Cluster NDB 6.4.0 and
later) ndbmtd binaries for the data node
processes.
There are many parameters which control buffer sizes, pool sizes, timeouts, and so forth. The only mandatory parameters are:
Either ExecuteOnComputer
or
HostName
, which must be defined in the
local [ndbd]
section.
The parameter
NoOfReplicas
, which
must be defined in the [ndbd default]
section, as it is common to all Cluster data nodes.
It is no longer strictly necessary to set
NoOfReplicas
starting
with MySQL Cluster NDB 6.3.25 and MySQL Cluster NDB 7.0.6,
where it acquires a default value (2). However, it remains
good practice to set it explicitly.
Most data node parameters are set in the [ndbd
default]
section. Only those parameters explicitly
stated as being able to set local values are permitted to be
changed in the [ndbd]
section. Where present,
HostName
, NodeId
and
ExecuteOnComputer
must
be defined in the local [ndbd]
section, and
not in any other section of config.ini
. In
other words, settings for these parameters are specific to one
data node.
For those parameters affecting memory usage or buffer sizes, it
is possible to use K
, M
,
or G
as a suffix to indicate units of 1024,
1024×1024, or 1024×1024×1024. (For example,
100K
means 100 × 1024 = 102400.)
Parameter names and values are currently case-sensitive.
Information about configuration parameters specific to MySQL Cluster Disk Data tables can be found later in this section (see Disk Data Configuration Parameters).
Beginning with MySQL Cluster NDB 6.4.0, all of these parameters
also apply to ndbmtd (the multi-threaded
version of ndbd). Two additional data node
configuration
parameters—MaxNoOfExecutionThreads
and
ThreadConfig
—apply
to ndbmtd only; these have no effect when
used with ndbd. For more information, see
Multi-Threading Configuration Parameters (ndbmtd).
See also Section 17.4.3, “ndbmtd — The MySQL Cluster Data Node Daemon (Multi-Threaded)”.
Identifying data nodes.
The NodeId
or Id
value
(that is, the data node identifier) can be allocated on the
command line when the node is started or in the configuration
file.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | unsigned | [none] | 1 - 48 | IS |
A unique node ID is used as the node's address for all cluster internal messages. For data nodes, this is an integer in the range 1 to 48 inclusive. Each node in the cluster must have a unique identifier.
NodeId
is the
preferred parameter name to use when identifying data nodes
beginning with MySQL Cluster NDB 6.2.19, MySQL Cluster NDB
6.3.39, MySQL Cluster NDB 7.0.20, and MySQL Cluster NDB
7.1.9. Although Id
continues to be
supported for backward compatibility, it is now deprecated
and generates a warning when used.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | unsigned | [none] | 1 - 48 | IS |
A unique node ID is used as the node's address for all cluster internal messages. For data nodes, this is an integer in the range 1 to 48 inclusive. Each node in the cluster must have a unique identifier.
NodeId
is the preferred parameter name to
use when identifying data nodes beginning with MySQL Cluster
NDB 6.2.19, MySQL Cluster NDB 6.3.39, MySQL Cluster NDB
7.0.20, and MySQL Cluster NDB 7.1.9. Although
Id
continues to be
supported for backward compatibility, it is now deprecated
and generates a warning when used.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | name | [none] | ... | S |
This refers to the Id
set for one of the
computers defined in a [computer]
section.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | name or IP address | localhost | ... | N |
Specifying this parameter defines the hostname of the
computer on which the data node is to reside. To specify a
hostname other than localhost
, either
this parameter or ExecuteOnComputer
is
required.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | unsigned | [none] | 1 - 64K | N |
Each node in the cluster uses a port to connect to other nodes. By default, this port is allocated dynamically in such a way as to ensure that no two nodes on the same host computer receive the same port number, so it should normally not be necessary to specify a value for this parameter.
However, if you need to be able to open specific ports in a
firewall to permit communication between data nodes and API
nodes (including SQL nodes), you can set this parameter to
the number of the desired port in an
[ndbd]
section or (if you need to do this
for multiple data nodes) the [ndbd
default]
section of the
config.ini
file, and then open the port
having that number for incoming connections from SQL nodes,
API nodes, or both.
Connections from data nodes to management nodes is done
using the ndb_mgmd management port (the
management server's PortNumber
;
see Section 17.3.2.5, “Defining a MySQL Cluster Management Server”) so
outgoing connections to that port from any data nodes
should always be permitted.
Setting this parameter to TRUE
or
1
binds IP_ADDR_ANY
so
that connections can be made from anywhere (for
autogenerated connections). The default is
FALSE
(0
).
This parameter was added in MySQL Cluster NDB 6.2.0.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.4.0 | [none] | 0 - 65536 | IS |
This parameter can be used to assign a data node to a
specific node group. It is read only when the cluster is
started for the first time, and cannot be used to reassign a
data node to a different node group online. It is generally
not desirable to use this parameter in the [ndbd
default]
section of the
config.ini
file, and care must be taken
not to assign nodes to node groups in such a way that an
invalid numbers of nodes are assigned to any node groups.
The NodeGroup
parameter is chiefly intended for use in adding a new node
group to a running MySQL Cluster without having to perform a
rolling restart. For this purpose, you should set it to
65536 (the maximum value). You are not required to set a
NodeGroup
value for
all cluster data nodes, only for those nodes which are to be
started and added to the cluster as a new node group at a
later time. For more information, see
Section 17.5.13.3, “Adding MySQL Cluster Data Nodes Online: Detailed Example”.
This parameter was added in MySQL Cluster NDB 6.4.0.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | [none] | 1 - 4 | IS |
NDB 6.3.25 | integer | 2 | 1 - 4 | IS |
NDB 7.0.6 | integer | 2 | 1 - 4 | IS |
This global parameter can be set only in the [ndbd
default]
section, and defines the number of
replicas for each table stored in the cluster. This
parameter also specifies the size of node groups. A node
group is a set of nodes all storing the same information.
Node groups are formed implicitly. The first node group is
formed by the set of data nodes with the lowest node IDs,
the next node group by the set of the next lowest node
identities, and so on. By way of example, assume that we
have 4 data nodes and that
NoOfReplicas
is set
to 2. The four data nodes have node IDs 2, 3, 4 and 5. Then
the first node group is formed from nodes 2 and 3, and the
second node group by nodes 4 and 5. It is important to
configure the cluster in such a manner that nodes in the
same node groups are not placed on the same computer because
a single hardware failure would cause the entire cluster to
fail.
If no node IDs are provided, the order of the data nodes
will be the determining factor for the node group. Whether
or not explicit assignments are made, they can be viewed in
the output of the management client's
SHOW
command.
Prior to MySQL Cluster NDB 6.3.25 and MySQL Cluster NDB
7.0.6, there was no default value for
NoOfReplicas
;
beginning with these versions, the default value is 2, which
is the recommended setting in most common usage scenarios.
(Bug #44746)
The maximum possible value is 4; currently, only the values 1 and 2 are actually supported.
Setting
NoOfReplicas
to 1
means that there is only a single copy of all Cluster
data; in this case, the loss of a single data node causes
the cluster to fail because there are no additional copies
of the data stored by that node.
The value for this parameter must divide evenly into the
number of data nodes in the cluster. For example, if there
are two data nodes, then
NoOfReplicas
must be
equal to either 1 or 2, since 2/3 and 2/4 both yield
fractional values; if there are four data nodes, then
NoOfReplicas
must be
equal to 1, 2, or 4.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | path | . | ... | IN |
This parameter specifies the directory where trace files, log files, pid files and error logs are placed.
The default is the data node process working directory.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | path | DataDir | ... | IN |
This parameter specifies the directory where all files
created for metadata, REDO logs, UNDO logs (for Disk Data
tables), and data files are placed. The default is the
directory specified by DataDir
.
This directory must exist before the ndbd process is initiated.
The recommended directory hierarchy for MySQL Cluster
includes /var/lib/mysql-cluster
, under
which a directory for the node's file system is created. The
name of this subdirectory contains the node ID. For example,
if the node ID is 2, this subdirectory is named
ndb_2_fs
.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | path | [see text] | ... | IN |
This parameter specifies the directory in which backups are placed.
The string '/BACKUP
' is always appended
to this value. For example, if you set the value of
BackupDataDir
to
/var/lib/cluster-data
, then all
backups are stored under
/var/lib/cluster-data/BACKUP
. This
also means that the effective default
backup location is the directory named
BACKUP
under the location specified
by the
FileSystemPath
parameter.
DataMemory
and
IndexMemory
are
[ndbd]
parameters specifying the size of
memory segments used to store the actual records and their
indexes. In setting values for these, it is important to
understand how
DataMemory
and
IndexMemory
are used, as
they usually need to be updated to reflect actual usage by the
cluster:
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | bytes | 80M | 1M - 1024G | N |
This parameter defines the amount of space (in bytes) available for storing database records. The entire amount specified by this value is allocated in memory, so it is extremely important that the machine has sufficient physical memory to accommodate it.
The memory allocated by
DataMemory
is used
to store both the actual records and indexes. There is a
16-byte overhead on each record; an additional amount for
each record is incurred because it is stored in a 32KB page
with 128 byte page overhead (see below). There is also a
small amount wasted per page due to the fact that each
record is stored in only one page.
For variable-size table attributes in MySQL 5.1, the data is
stored on separate data pages, allocated from
DataMemory
.
Variable-length records use a fixed-size part with an extra
overhead of 4 bytes to reference the variable-size part. The
variable-size part has 2 bytes overhead plus 2 bytes per
attribute.
Prior to MySQL Cluster NDB 7.0, the maximum record size was 8052 bytes; in MySQL Cluster NDB 7.0 and later, it is 14000 bytes.
The memory space defined by
DataMemory
is also
used to store ordered indexes, which use about 10 bytes per
record. Each table row is represented in the ordered index.
A common error among users is to assume that all indexes are
stored in the memory allocated by
IndexMemory
, but
this is not the case: Only primary key and unique hash
indexes use this memory; ordered indexes use the memory
allocated by
DataMemory
. However,
creating a primary key or unique hash index also creates an
ordered index on the same keys, unless you specify
USING HASH
in the index creation
statement. This can be verified by running ndb_desc
-d db_name
table_name
in the
management client.
Currently, MySQL Cluster can use a maximum of 512 MB for
hash indexes per partition, which means in some cases it is
possible to get Table is full errors
in MySQL client applications even when ndb_mgm -e
"ALL REPORT MEMORYUSAGE" shows significant free
DataMemory
. This can
also pose a problem with data node restarts on nodes that
are heavily loaded with data. You can force
NDB
to create extra partitions
for MySQL Cluster tables and thus have more memory available
for hash indexes by using the MAX_ROWS
option for CREATE TABLE
. In
general, setting MAX_ROWS
to twice the
number of rows that you expect to store in the table should
be sufficient. In MySQL Cluster 7.1.18 and later, you can
also use the
MinFreePct
configuration parameter to help avoid problems with node
restarts. (Bug #13436216)
The memory space allocated by
DataMemory
consists
of 32KB pages, which are allocated to table fragments. Each
table is normally partitioned into the same number of
fragments as there are data nodes in the cluster. Thus, for
each node, there are the same number of fragments as are set
in NoOfReplicas
.
In addition, due to the way in which new pages are allocated
when the capacity of the current page is exhausted, there is
an additional overhead of approximately 18.75%. When more
DataMemory
is
required, more than one new page is allocated, according to
the following formula:
number of new pages = FLOOR(number of current pages × 0.1875) + 1
For example, if 15 pages are currently allocated to a given
table and an insert to this table requires additional
storage space, the number of new pages allocated to the
table is FLOOR(15 × 0.1875) + 1 =
FLOOR(2.8125) + 1 = 2 + 1 =
3
. Now 15 + 3 = 18 memory pages are
allocated to the table. When the last of these 18 pages
becomes full, FLOOR(18 × 0.1875) + 1
= FLOOR(3.3750) + 1 = 3 + 1 =
4
new pages are allocated, so the total number of
pages allocated to the table is now 22.
The “18.75% + 1” overhead is no longer required beginning with MySQL Cluster NDB 6.2.3 and MySQL Cluster NDB 6.3.0.
Once a page has been allocated, it is currently not possible
to return it to the pool of free pages, except by deleting
the table. (This also means that
DataMemory
pages,
once allocated to a given table, cannot be used by other
tables.) Performing a node recovery also compresses the
partition because all records are inserted into empty
partitions from other live nodes.
The DataMemory
memory space also contains UNDO information: For each
update, a copy of the unaltered record is allocated in the
DataMemory
. There is
also a reference to each copy in the ordered table indexes.
Unique hash indexes are updated only when the unique index
columns are updated, in which case a new entry in the index
table is inserted and the old entry is deleted upon commit.
For this reason, it is also necessary to allocate enough
memory to handle the largest transactions performed by
applications using the cluster. In any case, performing a
few large transactions holds no advantage over using many
smaller ones, for the following reasons:
Large transactions are not any faster than smaller ones
Large transactions increase the number of operations that are lost and must be repeated in event of transaction failure
Large transactions use more memory
The default value for
DataMemory
is 80MB;
the minimum is 1MB. There is no maximum size, but in reality
the maximum size has to be adapted so that the process does
not start swapping when the limit is reached. This limit is
determined by the amount of physical RAM available on the
machine and by the amount of memory that the operating
system may commit to any one process. 32-bit operating
systems are generally limited to 2–4GB per process;
64-bit operating systems can use more. For large databases,
it may be preferable to use a 64-bit operating system for
this reason.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | bytes | 18M | 1M - 1T | N |
This parameter controls the amount of storage used for hash indexes in MySQL Cluster. Hash indexes are always used for primary key indexes, unique indexes, and unique constraints. When defining a primary key or a unique index, two indexes are created, one of which is a hash index used for all tuple accesses as well as lock handling. This index is also used to enforce unique constraints.
You can estimate the size of a hash index using this formula:
size = ( (fragments
* 32K) + (rows
* 18) ) *replicas
fragments
is the number of
fragments, replicas
is the number
of replicas (normally 2), and
rows
is the number of rows. If a
table has one million rows, 8 fragments, and 2 replicas, the
expected index memory usage is calculated as shown here:
((8 * 32K) + (1000000 * 18)) * 2 = ((8 * 32768) + (1000000 * 18)) * 2 = (262144 + 18000000) * 2 = 18262144 * 2 = 36524288 bytes = ~35MB
The default value for
IndexMemory
is 18MB.
The minimum is 1MB.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | % or bytes | 0 | 0 - 4294967039 (0xFFFFFEFF) | S |
MySQL 5.1.32 | % or bytes | 25 | 0 - 4294967039 (0xFFFFFEFF) | S |
MySQL 5.1.6 | % or bytes | 5 | 0 - 4294967039 (0xFFFFFEFF) | S |
NDB 7.2.1 | % or bytes | 25 | 0 - 4294967039 (0xFFFFFEFF) | S |
This parameter determines how much memory is allocated for
strings such as table names, and is specified in an
[ndbd]
or [ndbd
default]
section of the
config.ini
file. A value between
0
and 100
inclusive is
interpreted as a percent of the maximum default value, which
is calculated based on a number of factors including the
number of tables, maximum table name size, maximum size of
.FRM
files,
MaxNoOfTriggers
,
maximum column name size, and maximum default column value.
A value greater than 100
is interpreted
as a number of bytes.
The default value is 25—that is, 25 percent of the default maximum. (Previously, prior to MySQL Cluster NDB 6.2.18, MySQL Cluster NDB 6.3.24, and MySQL Cluster NDB 7.0.5, the default value was 5; prior to MySQL 5.1.6, the default was 0.)
Under most circumstances, the default value should be
sufficient, but when you have a great many Cluster tables
(1000 or more), it is possible to get Error 773
Out of string memory, please modify StringMemory
config parameter: Permanent error: Schema error,
in which case you should increase this value.
25
(25 percent) is not excessive, and
should prevent this error from recurring in all but the most
extreme conditions.
The following example illustrates how memory is used for a table. Consider this table definition:
CREATE TABLE example ( a INT NOT NULL, b INT NOT NULL, c INT NOT NULL, PRIMARY KEY(a), UNIQUE(b) ) ENGINE=NDBCLUSTER;
For each record, there are 12 bytes of data plus 12 bytes
overhead. Having no nullable columns saves 4 bytes of overhead.
In addition, we have two ordered indexes on columns
a
and b
consuming roughly
10 bytes each per record. There is a primary key hash index on
the base table using roughly 29 bytes per record. The unique
constraint is implemented by a separate table with
b
as primary key and a
as
a column. This other table consumes an additional 29 bytes of
index memory per record in the example
table
as well 8 bytes of record data plus 12 bytes of overhead.
Thus, for one million records, we need 58MB for index memory to handle the hash indexes for the primary key and the unique constraint. We also need 64MB for the records of the base table and the unique index table, plus the two ordered index tables.
You can see that hash indexes takes up a fair amount of memory space; however, they provide very fast access to the data in return. They are also used in MySQL Cluster to handle uniqueness constraints.
Currently, the only partitioning algorithm is hashing and ordered indexes are local to each node. Thus, ordered indexes cannot be used to handle uniqueness constraints in the general case.
An important point for both
IndexMemory
and
DataMemory
is that the
total database size is the sum of all data memory and all index
memory for each node group. Each node group is used to store
replicated information, so if there are four nodes with two
replicas, there will be two node groups. Thus, the total data
memory available is 2 ×
DataMemory
for each data
node.
It is highly recommended that
DataMemory
and
IndexMemory
be set to
the same values for all nodes. Data distribution is even over
all nodes in the cluster, so the maximum amount of space
available for any node can be no greater than that of the
smallest node in the cluster.
DataMemory
and
IndexMemory
can be
changed, but decreasing either of these can be risky; doing so
can easily lead to a node or even an entire MySQL Cluster that
is unable to restart due to there being insufficient memory
space. Increasing these values should be acceptable, but it is
recommended that such upgrades are performed in the same manner
as a software upgrade, beginning with an update of the
configuration file, and then restarting the management server
followed by restarting each data node in turn.
Beginning with MySQL Cluster NDB 7.0.29 and MySQL Cluster NDB
7.1.18, a proportion (5% by default) of data node resources
including DataMemory
and
IndexMemory
is kept in
reserve to insure that the data node does not exhaust its memory
when performing a restart. This can be adjusted using the
MinFreePct
data node
configuration parameter (default 5) introduced in the same
versions of MySQL Cluster.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 7.0.29 | unsigned | 5 | 0 - 100 | N |
NDB 7.1.18 | unsigned | 5 | 0 - 100 | N |
NDB 7.2.3 | unsigned | 5 | 0 - 100 | N |
Updates do not increase the amount of index memory used. Inserts take effect immediately; however, rows are not actually deleted until the transaction is committed.
Transaction parameters.
The next few [ndbd]
parameters that we
discuss are important because they affect the number of
parallel transactions and the sizes of transactions that can
be handled by the system.
MaxNoOfConcurrentTransactions
sets the number of parallel transactions possible in a node.
MaxNoOfConcurrentOperations
sets the number of records that can be in update phase or
locked simultaneously.
Both of these parameters (especially
MaxNoOfConcurrentOperations
)
are likely targets for users setting specific values and not
using the default value. The default value is set for systems
using small transactions, to ensure that these do not use
excessive memory.
MaxDMLOperationsPerTransaction
,
added in MySQL Cluster NDB 7.0.26 and MySQL Cluster NDB 7.1.15,
sets the maximum number of DML operations that can be performed
in a given transaction.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 4096 | 32 - 4294967039 (0xFFFFFEFF) | N |
Each cluster data node requires a transaction record for each active transaction in the cluster. The task of coordinating transactions is distributed among all of the data nodes. The total number of transaction records in the cluster is the number of transactions in any given node times the number of nodes in the cluster.
Transaction records are allocated to individual MySQL servers. Each connection to an SQL node requires at least one transaction record, plus an additional transaction object per table accessed by that connection. This means that a reasonable minimum for this parameter is
MaxNoOfConcurrentTransactions = (maximum number of tables accessed in any single transaction + 1) * number of cluster SQL nodes
Suppose that there are 10 SQL nodes using the cluster. A
single join involving 10 tables requires 11 transaction
records; if there are 10 such joins in a transaction, then
10 * 11 = 110 transaction records are required for this
transaction, per SQL node, or 110 * 10 = 1100 transaction
records total. Each data node can be expected to handle
TotalNoOfConcurrentTransactions / number of data nodes. For
a MySQL Cluster having 4 data nodes, this would mean setting
MaxNoOfConcurrentTransactions
on each
data node to 1100 / 4 = 275. In addition, you should provide
for failure recovery by ensuring that a single node group
can accommodate all concurrent transactions; in other words,
that each data node's MaxNoOfConcurrentTransactions is
sufficient to cover a number of transaction equal to
TotalNoOfConcurrentTransactions / number of node groups. If
this cluster has a single node group, then
MaxNoOfConcurrentTransactions
should be
set to 1100 (the same as the total number of concurrent
transactions for the entire cluster).
In addition, each transaction involves at least one
operation; for this reason, the value set for
MaxNoOfConcurrentTransactions
should
always be no more than the value of
MaxNoOfConcurrentOperations
.
This parameter must be set to the same value for all cluster data nodes. This is due to the fact that, when a data node fails, the oldest surviving node re-creates the transaction state of all transactions that were ongoing in the failed node.
It is possible to change this value using a rolling restart, but the amount of traffic on the cluster must be such that no more transactions occur than the lower of the old and new levels while this is taking place.
The default value is 4096.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 32K | 32 - 4294967039 (0xFFFFFEFF) | N |
It is a good idea to adjust the value of this parameter according to the size and number of transactions. When performing transactions which involve only a few operations and records, the default value for this parameter is usually sufficient. Performing large transactions involving many records usually requires that you increase its value.
Records are kept for each transaction updating cluster data, both in the transaction coordinator and in the nodes where the actual updates are performed. These records contain state information needed to find UNDO records for rollback, lock queues, and other purposes.
This parameter should be set to the number of records to be
updated simultaneously in transactions, divided by the
number of cluster data nodes. For example, in a cluster
which has four data nodes and which is expected to handle
one million concurrent updates using transactions, you
should set this value to 1000000 / 4 = 250000. To help
provide resiliency against failures, it is suggested that
you set this parameter to a value that is high enough to
permit an individual data node to handle the load for its
node group. In other words, you should set the value equal
to total number of concurrent operations / number
of node groups
. (In the case where there is a
single node group, this is the same as the total number of
concurrent operations for the entire cluster.)
Because each transaction always involves at least one
operation, the value of
MaxNoOfConcurrentOperations
should always
be greater than or equal to the value of
MaxNoOfConcurrentTransactions
.
Read queries which set locks also cause operation records to be created. Some extra space is allocated within individual nodes to accommodate cases where the distribution is not perfect over the nodes.
When queries make use of the unique hash index, there are actually two operation records used per record in the transaction. The first record represents the read in the index table and the second handles the operation on the base table.
The default value is 32768.
This parameter actually handles two values that can be configured separately. The first of these specifies how many operation records are to be placed with the transaction coordinator. The second part specifies how many operation records are to be local to the database.
A very large transaction performed on an eight-node cluster
requires as many operation records in the transaction
coordinator as there are reads, updates, and deletes
involved in the transaction. However, the operation records
of the are spread over all eight nodes. Thus, if it is
necessary to configure the system for one very large
transaction, it is a good idea to configure the two parts
separately. MaxNoOfConcurrentOperations
will always be used to calculate the number of operation
records in the transaction coordinator portion of the node.
It is also important to have an idea of the memory requirements for operation records. These consume about 1KB per record.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | UNDEFINED | 32 - 4294967039 (0xFFFFFEFF) | N |
By default, this parameter is calculated as 1.1 ×
MaxNoOfConcurrentOperations
.
This fits systems with many simultaneous transactions, none
of them being very large. If there is a need to handle one
very large transaction at a time and there are many nodes,
it is a good idea to override the default value by
explicitly specifying this parameter.
MaxDMLOperationsPerTransaction
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 7.0.26 | operations (DML) | 4294967295 | 32 - 4294967295 | N |
NDB 7.1.15 | operations (DML) | 4294967295 | 32 - 4294967295 | N |
Added in MySQL Cluster NDB 7.0.26 and MySQL Cluster NDB
7.1.15, this parameter limits the size of a transaction. The
transaction is aborted if it requires more than this many
DML operations. The minimum number of operations per
transaction is 32; however, you can set
MaxDMLOperationsPerTransaction
to 0 to
disable any limitation on the number of DML operations per
transaction. The maximum (and default) is 4294967295.
Transaction temporary storage.
The next set of [ndbd]
parameters is used
to determine temporary storage when executing a statement that
is part of a Cluster transaction. All records are released
when the statement is completed and the cluster is waiting for
the commit or rollback.
The default values for these parameters are adequate for most situations. However, users with a need to support transactions involving large numbers of rows or operations may need to increase these values to enable better parallelism in the system, whereas users whose applications require relatively small transactions can decrease the values to save memory.
MaxNoOfConcurrentIndexOperations
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 8K | 0 - 4294967039 (0xFFFFFEFF) | N |
For queries using a unique hash index, another temporary set
of operation records is used during a query's execution
phase. This parameter sets the size of that pool of records.
Thus, this record is allocated only while executing a part
of a query. As soon as this part has been executed, the
record is released. The state needed to handle aborts and
commits is handled by the normal operation records, where
the pool size is set by the parameter
MaxNoOfConcurrentOperations
.
The default value of this parameter is 8192. Only in rare cases of extremely high parallelism using unique hash indexes should it be necessary to increase this value. Using a smaller value is possible and can save memory if the DBA is certain that a high degree of parallelism is not required for the cluster.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 4000 | 0 - 4294967039 (0xFFFFFEFF) | N |
The default value of
MaxNoOfFiredTriggers
is 4000, which is sufficient for most situations. In some
cases it can even be decreased if the DBA feels certain the
need for parallelism in the cluster is not high.
A record is created when an operation is performed that affects a unique hash index. Inserting or deleting a record in a table with unique hash indexes or updating a column that is part of a unique hash index fires an insert or a delete in the index table. The resulting record is used to represent this index table operation while waiting for the original operation that fired it to complete. This operation is short-lived but can still require a large number of records in its pool for situations with many parallel write operations on a base table containing a set of unique hash indexes.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | bytes | 1M | 1K - 4294967039 (0xFFFFFEFF) | N |
The memory affected by this parameter is used for tracking operations fired when updating index tables and reading unique indexes. This memory is used to store the key and column information for these operations. It is only very rarely that the value for this parameter needs to be altered from the default.
The default value for
TransactionBufferMemory
is 1MB.
Normal read and write operations use a similar buffer, whose
usage is even more short-lived. The compile-time parameter
ZATTRBUF_FILESIZE
(found in
ndb/src/kernel/blocks/Dbtc/Dbtc.hpp
)
set to 4000 × 128 bytes (500KB). A similar buffer for
key information, ZDATABUF_FILESIZE
(also
in Dbtc.hpp
) contains 4000 × 16 =
62.5KB of buffer space. Dbtc
is the
module that handles transaction coordination.
Scans and buffering.
There are additional [ndbd]
parameters in
the Dblqh
module (in
ndb/src/kernel/blocks/Dblqh/Dblqh.hpp
)
that affect reads and updates. These include
ZATTRINBUF_FILESIZE
, set by default to
10000 × 128 bytes (1250KB) and
ZDATABUF_FILE_SIZE
, set by default to
10000*16 bytes (roughly 156KB) of buffer space. To date, there
have been neither any reports from users nor any results from
our own extensive tests suggesting that either of these
compile-time limits should be increased.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 256 | 2 - 500 | N |
This parameter is used to control the number of parallel
scans that can be performed in the cluster. Each transaction
coordinator can handle the number of parallel scans defined
for this parameter. Each scan query is performed by scanning
all partitions in parallel. Each partition scan uses a scan
record in the node where the partition is located, the
number of records being the value of this parameter times
the number of nodes. The cluster should be able to sustain
MaxNoOfConcurrentScans
scans concurrently from all nodes in the cluster.
Scans are actually performed in two cases. The first of these cases occurs when no hash or ordered indexes exists to handle the query, in which case the query is executed by performing a full table scan. The second case is encountered when there is no hash index to support the query but there is an ordered index. Using the ordered index means executing a parallel range scan. The order is kept on the local partitions only, so it is necessary to perform the index scan on all partitions.
The default value of
MaxNoOfConcurrentScans
is 256. The maximum value is 500.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | [see text] | 32 - 4294967039 (0xFFFFFEFF) | N |
NDB 7.2.0 | integer | [see text] | 32 - 4294967039 (0xFFFFFEFF) | N |
Specifies the number of local scan records if many scans are not fully parallelized. In MySQL Cluster NDB 7.2.0 and later, when the number of local scan records is not provided, it is calculated as shown here:
4 * MaxNoOfConcurrentScans
* [# data nodes] + 2
In MySQL Cluster NDB 7.1 and earlier, the default is
calculated as the product of
MaxNoOfConcurrentScans
and the number of data nodes, plus 2. The minimum value is
32.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 64 | 1 - 992 | N |
NDB 7.2.1 | integer | 256 | 1 - 992 | N |
This parameter is used to calculate the number of lock records used to handle concurrent scan operations.
BatchSizePerLocalScan
has a strong
connection to the
BatchSize
defined in
the SQL nodes.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | bytes | 1M | 512K - 4294967039 (0xFFFFFEFF) | N |
NDB 7.0.4 | bytes | 4M | 512K - 4294967039 (0xFFFFFEFF) | N |
NDB 7.2.0 | bytes | 4M | 512K - 4294967039 (0xFFFFFEFF) | N |
NDB 7.1.31 | bytes | 64M | 512K - 4294967039 (0xFFFFFEFF) | N |
This is an internal buffer used for passing messages within individual nodes and between nodes. In MySQL Cluster NDB 6.4.3 and earlier, the default is 1MB; beginning with MySQL Cluster NDB 7.0.4, this increased to 4MB; the default increased to 64MB in MySQL Cluster NDB 7.1.31.
This parameter seldom needs to be changed from the default. However, when replicating a MySQL Cluster using ndbmtd for the data nodes, you may need to increase this value to 8MB (or possibly more) to prevent data node instability, because ndbmtd uses much more of this resource than ndbd does. Beginning with MySQL Cluster NDB 7.0.13 and MySQL Cluster NDB 7.1.2, this should no longer be necessary when using ndbmtd with MySQL Cluster Replication (Bug #46914).
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 7.0.23 | bytes | 32 | 1 - 4294967039 (0xFFFFFEFF) | N |
NDB 7.1.12 | bytes | 32 | 1 - 4294967039 (0xFFFFFEFF) | N |
NDB 7.2.0 | bytes | 256 | 1 - 4294967039 (0xFFFFFEFF) | N |
Beginning with MySQL Cluster NDB 7.0.23 and MySQL Cluster
NDB 7.1.12, it is possible to configure the maximum number
of parallel scans (TUP
scans and
TUX
scans) allowed before they begin
queuing for serial handling. (Previously, the maximum number
of parallel scans per fragment was fixed at 32.) You can
increase this to take advantage of any unused CPU when
performing large number of scans in parallel and improve
their performance.
Beginning with MySQL Cluster NDB 7.2.0, the default value for this parameter is increased from 32 to 256.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.1.19 | unsigned | 32M | 1M - 1G | N |
NDB 6.2.3 | unsigned | 32M | 1M - 1G | N |
MySQL 5.1.20 | unsigned | 32M | 1M - 1G | N |
This is the maximum size of the memory unit to use when
allocating memory for tables. In cases where
NDB
gives Out of
memory errors, but it is evident by examining the
cluster logs or the output of DUMP 1000
(see
DUMP 1000) that all
available memory has not yet been used, you can increase the
value of this parameter (or
MaxNoOfTables
, or both)
to cause NDB
to make sufficient
memory available.
This parameter was introduced in MySQL 5.1.20, MySQL Cluster NDB 6.1.12 and MySQL Cluster NDB 6.2.3.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 7.0.38 | LDM threads | 240 | 0 - 3840 | N |
NDB 7.1.27 | LDM threads | 240 | 0 - 3840 | N |
Beginning with MySQL Cluster NDB 7.0.38 and MySQL Cluster NDB
7.1.26, the size of the table hash maps used by
NDB
is configurable using this
parameter; previously this value was hard-coded.
DefaultHashMapSize
can take any of three
possible values (0, 240, 3840). These values and their effects
are described in the following table:
Value | Description / Effect |
---|---|
0 | Use the lowest value set, if any, for this parameter among all data nodes and API nodes in the cluster; if it is not set on any data or API node, use the default value. |
240 | Original hash map size used by default in all MySQL Cluster NDB 7.1 and MySQL Cluster NDB 7.0 (and earlier) releases; effectively the only value prior to versions 7.0.38 and 7.1.26 (when this parameter was introduced). |
3840 | Larger hash map size available beginning with MySQL Cluster NDB 7.0.38 and MySQL Cluster NDB 7.1.26 |
The primary intended use for this parameter is to facilitate
upgrades and esecially downgrades between MySQL Cluster NDB 7.1
and later MySQL Cluster versions in which the larger hash map
size (3840) is the default; by setting this parameter to 240
prior to performing an upgrade, you can cause the cluster to
continue using the smaller size for table hash maps, in which
case the tables remain compatible with earlier versions.
DefaultHashMapSize
can be set for individual
data nodes, API nodes, or both, but setting it once only, in the
[ndbd default]
section of the
config.ini
file, is the recommended
practice. You can also set this parameter to 0, in which case
the default value is used.
After increasing this parameter, to have existing tables to take
advantage of the new size, you can run
ALTER
TABLE ... REORGANIZE PARTITION
on them, after which
they can use the larger hash map size. This is in addition to
performing a rolling restart, which makes the larger hash maps
available to new tables, but does not enable existing tables to
use them.
Decreasing this parameter online after any tables have been
created or modified with DefaultHashMapSize
equal to 3840 is not currently supported.
Logging and checkpointing.
The following [ndbd]
parameters control log
and checkpoint behavior.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 8 | 3 - 4294967039 (0xFFFFFEFF) | IN |
MySQL 5.1.0 | integer | 16 | 3 - 4294967039 (0xFFFFFEFF) | IN |
This parameter sets the number of REDO log files for the node, and thus the amount of space allocated to REDO logging. Because the REDO log files are organized in a ring, it is extremely important that the first and last log files in the set (sometimes referred to as the “head” and “tail” log files, respectively) do not meet. When these approach one another too closely, the node begins aborting all transactions encompassing updates due to a lack of room for new log records.
A REDO
log record is not removed until
the required number of local checkpoints has been completed
since that log record was inserted (prior to MySQL Cluster
NDB 6.3.8, this was 3 local checkpoints; in later versions
of MySQL Cluster, only 2 local checkpoints are necessary).
Checkpointing frequency is determined by its own set of
configuration parameters discussed elsewhere in this
chapter.
How these parameters interact and proposals for how to configure them are discussed in Section 17.3.2.12, “Configuring MySQL Cluster Parameters for Local Checkpoints”.
The default parameter value is 16, which by default means 16
sets of 4 16MB files for a total of 1024MB. Beginning with
MySQL Cluster NDB 6.1.1, the size of the individual log
files is configurable using the
FragmentLogFileSize
parameter. In scenarios requiring a great many updates, the
value for
NoOfFragmentLogFiles
may need to be set as high as 300 or even higher to provide
sufficient space for REDO logs.
If the checkpointing is slow and there are so many writes to
the database that the log files are full and the log tail
cannot be cut without jeopardizing recovery, all updating
transactions are aborted with internal error code 410
(Out of log file space temporarily
). This
condition prevails until a checkpoint has completed and the
log tail can be moved forward.
This parameter cannot be changed “on the
fly”; you must restart the node using
--initial
. If you wish to change this
value for all data nodes in a running cluster, you can do
so using a rolling node restart (using
--initial
when starting each data node).
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.1.11 | bytes | 16M | 4M - 1G | IN |
Setting this parameter enables you to control directly the
size of redo log files. This can be useful in situations
when MySQL Cluster is operating under a high load and it is
unable to close fragment log files quickly enough before
attempting to open new ones (only 2 fragment log files can
be open at one time); increasing the size of the fragment
log files gives the cluster more time before having to open
each new fragment log file. The default value for this
parameter is 16M.
FragmentLogFileSize
was added in MySQL Cluster NDB 6.1.11.
For more information about fragment log files, see the
description for
NoOfFragmentLogFiles
.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.3.19 | [see values] | SPARSE | SPARSE, FULL | IN |
By default, fragment log files are created sparsely when performing an initial start of a data node—that is, depending on the operating system and file system in use, not all bytes are necessarily written to disk. Beginning with MySQL Cluster NDB 6.3.19, it is possible to override this behavior and force all bytes to be written regardless of the platform and file system type being used by mean of this parameter.
InitFragmentLogFiles
takes one of two values:
SPARSE
. Fragment log files are
created sparsely. This is the default value.
FULL
. Force all bytes of the fragment
log file to be written to disk.
Depending on your operating system and file system, setting
InitFragmentLogFiles=FULL
may help
eliminate I/O errors on writes to the REDO log.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 40 | 20 - 4294967039 (0xFFFFFEFF) | N |
MySQL 5.1.16 | unsigned | 0 | 20 - 4294967039 (0xFFFFFEFF) | N |
This parameter sets a ceiling on how many internal threads to allocate for open files. Any situation requiring a change in this parameter should be reported as a bug.
The default value is 0. (Prior to MySQL 5.1.16, the default was 40.) However, the minimum value to which this parameter can be set is 20.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.9 | files | 27 | 20 - 4294967039 (0xFFFFFEFF) | N |
This parameter sets the initial number of internal threads to allocate for open files.
The default value is 27.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 25 | 0 - 4294967039 (0xFFFFFEFF) | N |
This parameter sets the maximum number of trace files that are kept before overwriting old ones. Trace files are generated when, for whatever reason, the node crashes.
The default is 25 trace files.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.3.23 | seconds | 0 | 0 - 600 | N |
NDB 6.4.3 | seconds | 0 | 0 - 600 | N |
In parallel data node recovery (supported in MySQL Cluster NDB 6.3.8 and later), only table data is actually copied and synchronized in parallel; synchronization of metadata such as dictionary and checkpoint information is done in a serial fashion. In addition, recovery of dictionary and checkpoint information cannot be executed in parallel with performing of local checkpoints. This means that, when starting or restarting many data nodes concurrently, data nodes may be forced to wait while a local checkpoint is performed, which can result in longer node recovery times.
Beginning with MySQL Cluster NDB 6.3.23 and MySQL Cluster NDB 6.4.3, it is possible to force a delay in the local checkpoint to permit more (and possibly all) data nodes to complete metadata synchronization; once each data node's metadata synchronization is complete, all of the data nodes can recover table data in parallel, even while the local checkpoint is being executed.
To force such a delay, you can set
MaxLCPStartDelay
,
which determines the number of seconds the cluster can wait
to begin a local checkpoint while data nodes continue to
synchronize metadata. This parameter should be set in the
[ndbd default]
section of the
config.ini
file, so that it is the same
for all data nodes. The maximum value is 600; the default is
0.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 7.0.40 | second | 60 | 0 - 4294967039 (0xFFFFFEFF) | N |
NDB 7.1.29 | second | 60 | 0 - 4294967039 (0xFFFFFEFF) | N |
A local checkpoint fragment scan watchdog checks
periodically for no progress in each fragment scan performed
as part of a local checkpoint, and shuts down the node if
there is no progress after a given amount of time has
elapsed. Prior to MySQL Cluster NDB 7.0.39 and MySQL Cluster
NDB 7.1.29, this interval is always 60 seconds (Bug
#16630410). In MySQL Cluster NDB 7.0.39, MySQL Cluster NDB
7.1.29, and later this interval can be set using the
LcpScanProgressTimeout
data node configuration parameter, which sets the maximum
time for which the local checkpoint can be stalled before
the LCP fragment scan watchdog shuts down the node.
The default value is 60 seconds (providing compatibility with previous releases). Setting this parameter to 0 disables the LCP fragment scan watchdog altogether.
Metadata objects.
The next set of [ndbd]
parameters defines
pool sizes for metadata objects, used to define the maximum
number of attributes, tables, indexes, and trigger objects
used by indexes, events, and replication between clusters.
Note that these act merely as “suggestions” to
the cluster, and any that are not specified revert to the
default values shown.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 1000 | 32 - 4294967039 (0xFFFFFEFF) | N |
This parameter sets a suggested maximum number of attributes
that can be defined in the cluster; like
MaxNoOfTables
, it is
not intended to function as a hard upper limit.
Prior to MySQL Cluster NDB 6.3.45, MySQL Cluster NDB 7.0.26,
and MySQL Cluster NDB 7.1.15, this parameter was sometimes
treated as a hard limit for certain operations. This caused
problems with MySQL Cluster Replication, when it was
possible to create more tables than could be replicated, and
sometimes led to confusion when it was possible (or not
possible, depending on the circumstances) to create more
than MaxNoOfAttributes
attributes. (Bug
#61684)
The default value is 1000, with the minimum possible value being 32. The maximum is 4294967039. Each attribute consumes around 200 bytes of storage per node due to the fact that all metadata is fully replicated on the servers.
When setting
MaxNoOfAttributes
,
it is important to prepare in advance for any
ALTER TABLE
statements that
you might want to perform in the future. This is due to the
fact, during the execution of ALTER
TABLE
on a Cluster table, 3 times the number of
attributes as in the original table are used, and a good
practice is to permit double this amount. For example, if
the MySQL Cluster table having the greatest number of
attributes
(greatest_number_of_attributes
)
has 100 attributes, a good starting point for the value of
MaxNoOfAttributes
would be 6 *
.
greatest_number_of_attributes
=
600
You should also estimate the average number of attributes
per table and multiply this by
MaxNoOfTables
. If
this value is larger than the value obtained in the previous
paragraph, you should use the larger value instead.
Assuming that you can create all desired tables without any
problems, you should also verify that this number is
sufficient by trying an actual ALTER
TABLE
after configuring the parameter. If this is
not successful, increase
MaxNoOfAttributes
by
another multiple of
MaxNoOfTables
and
test it again.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 128 | 8 - 20320 | N |
A table object is allocated for each table and for each
unique hash index in the cluster. This parameter sets a
suggested maximum number of table objects for the cluster as
a whole; like
MaxNoOfAttributes
,
it is not intended to function as a hard upper limit.
Prior to MySQL Cluster NDB 6.3.45, MySQL Cluster NDB 7.0.26,
and MySQL Cluster NDB 7.1.15, this parameter was sometimes
treated as a hard limit for certain operations. This caused
problems with MySQL Cluster Replication, when it was
possible to create more tables than could be replicated, and
sometimes led to confusion when it was possible (or not
possible, depending on the circumstances) to create more
than MaxNoOfTables
tables.
For each attribute that has a
BLOB
data type an extra table
is used to store most of the
BLOB
data. These tables also
must be taken into account when defining the total number of
tables.
The default value of this parameter is 128. The minimum is 8 and the maximum is 20320. Each table object consumes approximately 20KB per node.
The sum of
MaxNoOfTables
,
MaxNoOfOrderedIndexes
,
and
MaxNoOfUniqueHashIndexes
must not exceed 232
– 2
(4294967294).
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 128 | 0 - 4294967039 (0xFFFFFEFF) | N |
For each ordered index in the cluster, an object is
allocated describing what is being indexed and its storage
segments. By default, each index so defined also defines an
ordered index. Each unique index and primary key has both an
ordered index and a hash index.
MaxNoOfOrderedIndexes
sets the total number of ordered indexes that can be in use
in the system at any one time.
The default value of this parameter is 128. Each index object consumes approximately 10KB of data per node.
The sum of
MaxNoOfTables
,
MaxNoOfOrderedIndexes
,
and
MaxNoOfUniqueHashIndexes
must not exceed 232
– 2
(4294967294).
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 64 | 0 - 4294967039 (0xFFFFFEFF) | N |
For each unique index that is not a primary key, a special
table is allocated that maps the unique key to the primary
key of the indexed table. By default, an ordered index is
also defined for each unique index. To prevent this, you
must specify the USING HASH
option when
defining the unique index.
The default value is 64. Each index consumes approximately 15KB per node.
The sum of
MaxNoOfTables
,
MaxNoOfOrderedIndexes
,
and
MaxNoOfUniqueHashIndexes
must not exceed 232
– 2
(4294967294).
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 768 | 0 - 4294967039 (0xFFFFFEFF) | N |
Internal update, insert, and delete triggers are allocated for each unique hash index. (This means that three triggers are created for each unique hash index.) However, an ordered index requires only a single trigger object. Backups also use three trigger objects for each normal table in the cluster.
Replication between clusters also makes use of internal triggers.
This parameter sets the maximum number of trigger objects in the cluster.
The default value is 768.
This parameter is deprecated. You should use
MaxNoOfOrderedIndexes
and
MaxNoOfUniqueHashIndexes
instead.
This parameter is used only by unique hash indexes. There needs to be one record in this pool for each unique hash index defined in the cluster.
The default value of this parameter is 128.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.2.10 | unsigned | 0 | 0 - 4294967039 (0xFFFFFEFF) | N |
NDB 6.3.7 | unsigned | 0 | 0 - 4294967039 (0xFFFFFEFF) | N |
Each NDB
table in a MySQL
Cluster requires a subscription in the NDB kernel. For some
NDB API applications, it may be necessary or desirable to
change this parameter, which became available in MySQL
Cluster NDB 6.2.10 and MySQL Cluster NDB 6.3.7. However, for
normal usage with MySQL servers acting as SQL nodes, there
is not any need to do so.
The default value for
MaxNoOfSubscriptions
is 0, which is treated as equal to
MaxNoOfTables
. Each
subscription consumes 108 bytes.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.2.10 | unsigned | 0 | 0 - 4294967039 (0xFFFFFEFF) | N |
NDB 6.3.7 | unsigned | 0 | 0 - 4294967039 (0xFFFFFEFF) | N |
This parameter, added in MySQL Cluster NDB 6.2.10 and MySQL
Cluster NDB 6.3.7, is of interest only when using MySQL
Cluster Replication. The default value is 0, which is
treated as 2 * MaxNoOfTables
; that is,
there is one subscription per
NDB
table for each of two MySQL
servers (one acting as the replication master and the other
as the slave). Each subscriber uses 16 bytes of memory.
When using circular replication, multi-master replication,
and other replication setups involving more than 2 MySQL
servers, you should increase this parameter to the number of
mysqld processes included in replication
(this is often, but not always, the same as the number of
clusters). For example, if you have a circular replication
setup using three MySQL Clusters, with one
mysqld attached to each cluster, and each
of these mysqld processes acts as a
master and as a slave, you should set
MaxNoOfSubscribers
equal to 3 * MaxNoOfTables
.
For more information, see Section 17.6, “MySQL Cluster Replication”.
MaxNoOfConcurrentSubOperations
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.2.10 | unsigned | 256 | 0 - 4294967039 (0xFFFFFEFF) | N |
NDB 6.3.7 | unsigned | 256 | 0 - 4294967039 (0xFFFFFEFF) | N |
This parameter sets a ceiling on the number of operations that can be performed by all API nodes in the cluster at one time. The default value (256) is sufficient for normal operations, and might need to be adjusted only in scenarios where there are a great many API nodes each performing a high volume of operations concurrently.
This parameter was added in MySQL Cluster NDB 6.2.10 and MySQL Cluster NDB 6.3.7.
Boolean parameters.
The behavior of data nodes is also affected by a set of
[ndbd]
parameters taking on boolean values.
These parameters can each be specified as
TRUE
by setting them equal to
1
or Y
, and as
FALSE
by setting them equal to
0
or N
.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.4.0 | numeric | 1 | 0 - 1 | N |
Allocate memory for this data node after a connection to the management server has been established. Enabled by default.
Added in MySQL Cluster NDB 6.4.0.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | numeric | 0 | 0 - 2 | N |
MySQL 5.1.0 | true|false (1|0) | 0 | 0 - 1 | N |
MySQL 5.1.15 | numeric | 0 | 0 - 2 | N |
For a number of operating systems, including Solaris and Linux, it is possible to lock a process into memory and so avoid any swapping to disk. This can be used to help guarantee the cluster's real-time characteristics.
Beginning with MySQL 5.1.15 and MySQL Cluster NDB 6.1.1,
this parameter takes one of the integer values
0
, 1
, or
2
, which act as follows:
0
: Disables locking. This is the
default value.
1
: Performs the lock after allocating
memory for the process.
2
: Performs the lock before memory
for the process is allocated.
Previously, this parameter was a Boolean.
0
or false
was the
default setting, and disabled locking. 1
or true
enabled locking of the process
after its memory was allocated.
Beginning with MySQL 5.1.15 and MySQL Cluster NDB 6.1.1,
it is no longer possible to use true
or
false
for the value of this parameter;
when upgrading from a previous version, you must change
the value to 0
, 1
,
or 2
.
Prior to MySQL Cluster NDB 6.3.31 and MySQL Cluster NDB 7.0.11, setting this parameter did not cause the stated memory to be allocated when the node was started, but rather only when the memory was used by the data node process for other reasons. (Bug #37430)
If the operating system is not configured to permit
unprivileged users to lock pages, then the data node
process making use of this parameter may have to be run as
system root.
(LockPagesInMainMemory
uses the mlockall
function. From Linux
kernel 2.6.9, unprivileged users can lock memory as
limited by max locked memory
. For more
information, see ulimit -l and
http://linux.die.net/man/2/mlock).
Beginning with glibc
2.10,
glibc
uses per-thread arenas to reduce
lock contention on a shared pool, which consumes real
memory. In general, a data node process does not need
per-thread arenas, since it does not perform any memory
allocation after startup. (This difference in allocators
does not appear to affect performance significantly.)
The glibc
behavior is intended to be
configurable via the MALLOC_ARENA_MAX
environment variable, but a bug in this mechanism prior to
glibc
2.16 meant that this variable
could not be set to less than 8, so that the wasted memory
could not be reclaimed. (Bug #15907219; see also
http://sourceware.org/bugzilla/show_bug.cgi?id=13137
for more information concerning this issue.)
One possible workaround for this problem is to use the
LD_PRELOAD
environment variable to
preload a jemalloc
memory allocation
library to take the place of that supplied with
glibc
.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | boolean | 1 | 0, 1 | N |
This parameter specifies whether a data node process should exit or perform an automatic restart when an error condition is encountered.
This parameter's default value is 1
;
this means that, by default, an error causes the data node
process to halt.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 7.0.28, 5.1.56-ndb-7.1.17 | boolean | false | true, false | S |
NDB 7.2.1 | boolean | true | true, false | S |
This parameter was introduced in MySQL Cluster NDB 7.0.28 and MySQL Cluster NDB 7.1.17. When enabled, it forces a data node to shut down whenever it encounters a corrupted tuple. By default, it is disabled.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | true|false (1|0) | false | true, false | IS |
It is possible to specify MySQL Cluster tables as diskless, meaning that tables are not checkpointed to disk and that no logging occurs. Such tables exist only in main memory. A consequence of using diskless tables is that neither the tables nor the records in those tables survive a crash. However, when operating in diskless mode, it is possible to run ndbd on a diskless computer.
This feature causes the entire cluster to operate in diskless mode.
When this feature is enabled, Cluster online backup is disabled. In addition, a partial start of the cluster is not possible.
Diskless
is disabled
by default.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.1.11 | boolean | false | true, false | N |
NDB 6.2.3 | boolean | false | true, false | N |
NDB 6.3.0 | boolean | false | true, false | N |
Enabling this parameter causes
NDB
to attempt using
O_DIRECT
writes for LCP, backups, and
redo logs, often lowering kswapd and CPU
usage. When using MySQL Cluster on Linux, enable
ODirect
if you are
using a 2.6 or later kernel.
This parameter was added in MySQL 5.1.20, MySQL Cluster NDB 6.1.11, MySQL Cluster NDB 6.2.3, and MySQL Cluster NDB 6.3.0.
ODirect
is disabled
by default.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | error code | 2 | 0 - 4 | N |
This feature is accessible only when building the debug version where it is possible to insert errors in the execution of individual blocks of code as part of testing.
This feature is disabled by default.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.3.7 | boolean | false | true, false | N |
Setting this parameter to 1
causes backup
files to be compressed. The compression used is equivalent
to gzip --fast, and can save 50% or more
of the space required on the data node to store uncompressed
backup files. Compressed backups can be enabled for
individual data nodes, or for all data nodes (by setting
this parameter in the [ndbd default]
section of the config.ini
file).
You cannot restore a compressed backup to a cluster running a MySQL version that does not support this feature.
The default value is 0
(disabled).
This parameter was introduced in MySQL Cluster NDB 6.3.7.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.3.7 | boolean | false | true, false | N |
Setting this parameter to 1
causes local
checkpoint files to be compressed. The compression used is
equivalent to gzip --fast, and can save
50% or more of the space required on the data node to store
uncompressed checkpoint files. Compressed LCPs can be
enabled for individual data nodes, or for all data nodes (by
setting this parameter in the [ndbd
default]
section of the
config.ini
file).
You cannot restore a compressed local checkpoint to a cluster running a MySQL version that does not support this feature.
The default value is 0
(disabled).
This parameter was introduced in MySQL Cluster NDB 6.3.7.
There are a number of [ndbd]
parameters
specifying timeouts and intervals between various actions in
Cluster data nodes. Most of the timeout values are specified in
milliseconds. Any exceptions to this are mentioned where
applicable.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | milliseconds | 6000 | 70 - 4294967039 (0xFFFFFEFF) | N |
To prevent the main thread from getting stuck in an endless loop at some point, a “watchdog” thread checks the main thread. This parameter specifies the number of milliseconds between checks. If the process remains in the same state after three checks, the watchdog thread terminates it.
This parameter can easily be changed for purposes of experimentation or to adapt to local conditions. It can be specified on a per-node basis although there seems to be little reason for doing so.
The default timeout is 6000 milliseconds (6 seconds).
TimeBetweenWatchDogCheckInitial
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.20 | milliseconds | 6000 | 70 - 4294967039 (0xFFFFFEFF) | N |
This is similar to the
TimeBetweenWatchDogCheck
parameter, except that
TimeBetweenWatchDogCheckInitial
controls the amount of time that passes between execution
checks inside a database node in the early start phases
during which memory is allocated.
The default timeout is 6000 milliseconds (6 seconds).
This parameter was added in MySQL 5.1.20.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | milliseconds | 30000 | 0 - 4294967039 (0xFFFFFEFF) | N |
This parameter specifies how long the Cluster waits for all data nodes to come up before the cluster initialization routine is invoked. This timeout is used to avoid a partial Cluster startup whenever possible.
This parameter is overridden when performing an initial start or initial restart of the cluster.
The default value is 30000 milliseconds (30 seconds). 0 disables the timeout, in which case the cluster may start only if all nodes are available.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | milliseconds | 60000 | 0 - 4294967039 (0xFFFFFEFF) | N |
If the cluster is ready to start after waiting for
StartPartialTimeout
milliseconds but is still possibly in a partitioned state,
the cluster waits until this timeout has also passed. If
StartPartitionedTimeout
is set to 0, the cluster waits indefinitely.
This parameter is overridden when performing an initial start or initial restart of the cluster.
The default timeout is 60000 milliseconds (60 seconds).
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | milliseconds | 0 | 0 - 4294967039 (0xFFFFFEFF) | N |
If a data node has not completed its startup sequence within the time specified by this parameter, the node startup fails. Setting this parameter to 0 (the default value) means that no data node timeout is applied.
For nonzero values, this parameter is measured in milliseconds. For data nodes containing extremely large amounts of data, this parameter should be increased. For example, in the case of a data node containing several gigabytes of data, a period as long as 10–15 minutes (that is, 600000 to 1000000 milliseconds) might be required to perform a node restart.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 7.0.24 | milliseconds | 15000 | 0 - 4294967039 (0xFFFFFEFF) | N |
When a data node is configured with
Nodegroup = 65536
,
is regarded as not being assigned to any node group. When
that is done, the cluster waits
StartNoNodegroupTimeout
milliseconds,
then treats such nodes as though they had been added to the
list passed to the
--nowait-nodes
option, and
starts. The default value is 15000
(that
is, the management server waits 15 seconds). Setting this
parameter equal to 0
means that the
cluster waits indefinitely.
StartNoNodegroupTimeout
must be the same
for all data nodes in the cluster; for this reason, you
should always set it in the [ndbd
default]
section of the
config.ini
file, rather than for
individual data nodes.
This parameter was added in MySQL Cluster NDB 7.0.24 and MySQL Cluster NDB 7.1.13. See Section 17.5.13, “Adding MySQL Cluster Data Nodes Online”, for more information.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | milliseconds | 1500 | 10 - 4294967039 (0xFFFFFEFF) | N |
NDB 7.2.0 | milliseconds | 5000 | 10 - 4294967039 (0xFFFFFEFF) | N |
One of the primary methods of discovering failed nodes is by the use of heartbeats. This parameter states how often heartbeat signals are sent and how often to expect to receive them. After missing three heartbeat intervals in a row, the node is declared dead. Thus, the maximum time for discovering a failure through the heartbeat mechanism is four times the heartbeat interval.
In MySQL Cluster NDB 7.2.0 and later, the default heartbeat interval is 5000 milliseconds (5 seconds). Previously, the default was 1500 milliseconds (1.5 seconds). This parameter must not be changed drastically and should not vary widely between nodes. If one node uses 5000 milliseconds and the node watching it uses 1000 milliseconds, obviously the node will be declared dead very quickly. This parameter can be changed during an online software upgrade, but only in small increments.
See also Network communication and latency.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | milliseconds | 1500 | 100 - 4294967039 (0xFFFFFEFF) | N |
Each data node sends heartbeat signals to each MySQL server
(SQL node) to ensure that it remains in contact. If a MySQL
server fails to send a heartbeat in time it is declared
“dead,” in which case all ongoing transactions
are completed and all resources released. The SQL node
cannot reconnect until all activities initiated by the
previous MySQL instance have been completed. The
three-heartbeat criteria for this determination are the same
as described for
HeartbeatIntervalDbDb
.
The default interval is 1500 milliseconds (1.5 seconds). This interval can vary between individual data nodes because each data node watches the MySQL servers connected to it, independently of all other data nodes.
For more information, see Network communication and latency.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.3.35 | numeric | 0 | 0 - 65535 | S |
NDB 7.0.16 | numeric | 0 | 0 - 65535 | S |
NDB 7.1.5 | numeric | 0 | 0 - 65535 | S |
Data nodes send heartbeats to one another in a circular fashion whereby each data node monitors the previous one. If a heartbeat is not detected by a given data node, this node declares the previous data node in the circle “dead” (that is, no longer accessible by the cluster). The determination that a data node is dead is done globally; in other words; once a data node is declared dead, it is regarded as such by all nodes in the cluster.
It is possible for heartbeats between data nodes residing on different hosts to be too slow compared to heartbeats between other pairs of nodes (for example, due to a very low heartbeat interval or temporary connection problem), such that a data node is declared dead, even though the node can still function as part of the cluster.
In this type of situation, it may be that the order in which heartbeats are transmitted between data nodes makes a difference as to whether or not a particular data node is declared dead. If this declaration occurs unnecessarily, this can in turn lead to the unnecessary loss of a node group and as thus to a failure of the cluster.
Consider a setup where there are 4 data nodes A, B, C, and D
running on 2 host computers host1
and
host2
, and that these data nodes make up
2 node groups, as shown in the following table:
Node Group |
Nodes Running on | Nodes Running on host2 |
---|---|---|
Node Group 0: | Node A | Node B |
Node Group 1: | Node C | Node D |
Suppose the heartbeats are transmitted in the order A->B->C->D->A. In this case, the loss of the heartbeat between the hosts causes node B to declare node A dead and node C to declare node B dead. This results in loss of Node Group 0, and so the cluster fails. On the other hand, if the order of transmission is A->B->D->C->A (and all other conditions remain as previously stated), the loss of the heartbeat causes nodes A and D to be declared dead; in this case, each node group has one surviving node, and the cluster survives.
Priot to MySQL Cluster NDB 6.3.35, MySQL Cluster NDB 7.0.16,
and MySQL Cluster NDB 7.1.5, the order of heartbeat
transmission between data nodes was always automatically
determined by NDB
. However, beginning
with these versions, the
HeartbeatOrder
configuration parameter makes the order of heartbeat
transmission user-configurable.
The default value for
HeartbeatOrder
is
zero; allowing the default value to be used on all data
nodes causes the order of heartbeat transmission to be
determined by NDB
. If this parameter is
used, it must be set to a nonzero value (maximum 65535) for
every data node in the cluster, and this value must be
unique for each data node; this causes the heartbeat
transmission to proceed from data node to data node in the
order of their
HeartbeatOrder
values from lowest to highest (and then directly from the
data node having the highest
HeartbeatOrder
to
the data node having the lowest value, to complete the
circle). The values need not be consecutive; for example, to
force the heartbeat transmission order
A->B->D->C->A in the scenario outlined
previously, you could set the
HeartbeatOrder
values as shown here:
Node | HeartbeatOrder |
---|---|
A | 10 |
B | 20 |
C | 30 |
D | 25 |
To use this parameter to change the heartbeat transmission
order in a running MySQL Cluster, you must first set
HeartbeatOrder
for
each data node in the cluster in the global configuration
(config.ini
) file (or files). To cause
the change to take effect, you must perform either of the
following:
A complete shutdown and restart of the entire cluster.
2 rolling restarts of the cluster in succession. All nodes must be restarted in the same order in both rolling restarts.
You can use DUMP 908
to observe the
effect of this parameter in the data node logs.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 7.1.13 | string | 1500 | 0 - 4294967039 (0xFFFFFEFF) | N |
NDB 7.2.1 | string | 0 | 0 - 4294967039 (0xFFFFFEFF) | N |
Enables connection checking between data nodes. A data node
that fails to respond within an interval of
ConnectCheckIntervalDelay
seconds is
considered suspect, and is considered dead after two such
intervals.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | number of 4-byte words, as a base-2 logarithm | 20 | 0 - 31 | N |
This parameter is an exception in that it does not specify a time to wait before starting a new local checkpoint; rather, it is used to ensure that local checkpoints are not performed in a cluster where relatively few updates are taking place. In most clusters with high update rates, it is likely that a new local checkpoint is started immediately after the previous one has been completed.
The size of all write operations executed since the start of the previous local checkpoints is added. This parameter is also exceptional in that it is specified as the base-2 logarithm of the number of 4-byte words, so that the default value 20 means 4MB (4 × 220) of write operations, 21 would mean 8MB, and so on up to a maximum value of 31, which equates to 8GB of write operations.
All the write operations in the cluster are added together.
Setting
TimeBetweenLocalCheckpoints
to 6 or less means that local checkpoints will be executed
continuously without pause, independent of the cluster's
workload.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | milliseconds | 2000 | 10 - 32000 | N |
NDB 6.2.0 | milliseconds | 2000 | 20 - 32000 | N |
When a transaction is committed, it is committed in main memory in all nodes on which the data is mirrored. However, transaction log records are not flushed to disk as part of the commit. The reasoning behind this behavior is that having the transaction safely committed on at least two autonomous host machines should meet reasonable standards for durability.
It is also important to ensure that even the worst of cases—a complete crash of the cluster—is handled properly. To guarantee that this happens, all transactions taking place within a given interval are put into a global checkpoint, which can be thought of as a set of committed transactions that has been flushed to disk. In other words, as part of the commit process, a transaction is placed in a global checkpoint group. Later, this group's log records are flushed to disk, and then the entire group of transactions is safely committed to disk on all computers in the cluster.
This parameter defines the interval between global checkpoints. The default is 2000 milliseconds.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.2.5 | milliseconds | 100 | 0 - 32000 | N |
This parameter defines the interval between synchronization epochs for MySQL Cluster Replication. The default value is 100 milliseconds.
TimeBetweenEpochs
is
part of the implementation of “micro-GCPs”,
which can be used to improve the performance of MySQL
Cluster Replication. This parameter was introduced in MySQL
Cluster NDB 6.2.5 and MySQL Cluster NDB 6.3.2.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.22 | milliseconds | 4000 | 0 - 32000 | N |
MySQL 5.1.51 | milliseconds | 0 | 0 - 256000 | N |
This parameter defines a timeout for synchronization epochs for MySQL Cluster Replication. If a node fails to participate in a global checkpoint within the time determined by this parameter, the node is shut down. In MySQL Cluster NDB 7.2.0 and later, the default value is 0; in other words, the timeout is disabled. This represents a change from previous versions of MySQL Cluster, in which the default value was 4000 milliseconds (4 seconds).
TimeBetweenEpochsTimeout
is part of the implementation of “micro-GCPs”,
which can be used to improve the performance of MySQL
Cluster Replication. This parameter was introduced in MySQL
Cluster NDB 6.2.7 and MySQL Cluster NDB 6.3.4.
The following changes regarding this parameter were made in MySQL Cluster NDB 7.0.21 and MySQL Cluster NDB 7.1.10:
The maximum possible value for this parameter was increased from 32000 milliseconds to 256000 milliseconds.
Setting this parameter to zero now has the effect of disabling GCP stops caused by save timeouts, commit timeouts, or both.
The current value of this parameter and a warning are now written to the cluster log whenever a GCP save takes longer than 1 minute or a GCP save takes longer than 10 seconds.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.2.14 | epochs | 100 | 0 - 100000 | N |
The number of unprocessed epochs by which a subscribing node can lag behind. Exceeding this number causes a lagging subscriber to be disconnected.
The default value of 100 is sufficient for most normal
operations. If a subscribing node does lag enough to cause
disconnections, it is usually due to network or scheduling
issues with regard to processes or threads. (In rare
circumstances, the problem may be due to a bug in the
NDB
client.) It may be
desirable to set the value lower than the default when
epochs are longer.
Disconnection prevents client issues from affecting the data node service, running out of memory to buffer data, and eventually shutting down. Instead, only the client is affected as a result of the disconnect (by, for example gap events in the binary log), forcing the client to reconnect or restart the process.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.3.52 | bytes | 26214400 | 26214400 (0x01900000) - 4294967039 (0xFFFFFEFF) | N |
NDB 7.0.38 | bytes | 26214400 | 26214400 (0x01900000) - 4294967039 (0xFFFFFEFF) | N |
NDB 7.1.27 | bytes | 26214400 | 26214400 (0x01900000) - 4294967039 (0xFFFFFEFF) | N |
The total number of bytes allocated for buffering epochs by this node.
This parameter was introduced in MySQL Cluster NDB 6.3.52, MySQL Cluster NDB 7.0.38, and MySQL Cluster NDB 7.1.27. (Bug #16203623)
TimeBetweenInactiveTransactionAbortCheck
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | milliseconds | 1000 | 1000 - 4294967039 (0xFFFFFEFF) | N |
Timeout handling is performed by checking a timer on each transaction once for every interval specified by this parameter. Thus, if this parameter is set to 1000 milliseconds, every transaction will be checked for timing out once per second.
The default value is 1000 milliseconds (1 second).
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | milliseconds | [see text] | 0 - 4294967039 (0xFFFFFEFF) | N |
This parameter states the maximum time that is permitted to lapse between operations in the same transaction before the transaction is aborted.
The default for this parameter is 4G
(also the maximum). For a real-time database that needs to
ensure that no transaction keeps locks for too long, this
parameter should be set to a relatively small value. The
unit is milliseconds.
TransactionDeadlockDetectionTimeout
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | milliseconds | 1200 | 50 - 4294967039 (0xFFFFFEFF) | N |
When a node executes a query involving a transaction, the node waits for the other nodes in the cluster to respond before continuing. A failure to respond can occur for any of the following reasons:
The node is “dead”
The operation has entered a lock queue
The node requested to perform the action could be heavily overloaded.
This timeout parameter states how long the transaction coordinator waits for query execution by another node before aborting the transaction, and is important for both node failure handling and deadlock detection. In MySQL 5.1.10 and earlier versions, setting it too high could cause undesirable behavior in situations involving deadlocks and node failure. Beginning with MySQL 5.1.11, active transactions occurring during node failures are actively aborted by the MySQL Cluster Transaction Coordinator, and so high settings are no longer an issue with this parameter.
The default timeout value is 1200 milliseconds (1.2 seconds).
Prior to MySQL Cluster NDB versions 6.2.18, 6.3.24, and 7.0.5, the effective minimum for this parameter was 100 milliseconds. (Bug #44099) Beginning with these versions, the actual minimum is 50 milliseconds.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.12 | bytes | 4M | 32K - 4294967039 (0xFFFFFEFF) | N |
This is the maximum number of bytes to store before flushing
data to a local checkpoint file. This is done to prevent
write buffering, which can impede performance significantly.
This parameter is not intended to take
the place of
TimeBetweenLocalCheckpoints
.
When ODirect
is
enabled, it is not necessary to set
DiskSyncSize
; in
fact, in such cases its value is simply ignored.
The default value is 4M (4 megabytes).
This parameter was added in MySQL 5.1.12.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.12 | bytes | 10M | 1M - 4294967039 (0xFFFFFEFF) | N |
The amount of data,in bytes per second, that is sent to disk during a local checkpoint. This allocation is shared by DML operations and backups (but not backup logging), which means that backups started during times of intensive DML may be impaired by flooding of the redo log buffer and may fail altogether if the contention is sufficiently severe.
The default value is 10M (10 megabytes per second).
This parameter was added in MySQL 5.1.12.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | bytes | 100M | 1M - 4294967039 (0xFFFFFEFF) | N |
The amount of data,in bytes per second, that is sent to disk during a local checkpoint as part of a restart operation.
The default value is 100M (100 megabytes per second).
This parameter was added in MySQL 5.1.12.
NoOfDiskPagesToDiskAfterRestartTUP
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | 8K pages/100 milliseconds | 40 | 1 - 4294967039 (0xFFFFFEFF) | N |
When executing a local checkpoint, the algorithm flushes all
data pages to disk. Merely doing so as quickly as possible
without any moderation is likely to impose excessive loads
on processors, networks, and disks. To control the write
speed, this parameter specifies how many pages per 100
milliseconds are to be written. In this context, a
“page” is defined as 8KB. This parameter is
specified in units of 80KB per second, so setting
NoOfDiskPagesToDiskAfterRestartTUP
to a value of 20
entails writing 1.6MB in
data pages to disk each second during a local checkpoint.
This value includes the writing of UNDO log records for data
pages. That is, this parameter handles the limitation of
writes from data memory. (See the entry for
IndexMemory
for
information about index pages.)
In short, this parameter specifies how quickly to execute
local checkpoints. It operates in conjunction with
NoOfFragmentLogFiles
,
DataMemory
, and
IndexMemory
.
For more information about the interaction between these parameters and possible strategies for choosing appropriate values for them, see Section 17.3.2.12, “Configuring MySQL Cluster Parameters for Local Checkpoints”.
The default value is 40 (3.2MB of data pages per second).
This parameter is deprecated as of MySQL 5.1.6. For MySQL
5.1.12 and later versions, use
DiskCheckpointSpeed
and DiskSyncSize
instead.
NoOfDiskPagesToDiskAfterRestartACC
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | 8K pages/100 milliseconds | 20 | 1 - 4294967039 (0xFFFFFEFF) | N |
This parameter uses the same units as
NoOfDiskPagesToDiskAfterRestartTUP
and acts in a similar fashion, but limits the speed of
writing index pages from index memory.
The default value of this parameter is 20 (1.6MB of index memory pages per second).
This parameter is deprecated as of MySQL 5.1.6. For MySQL
5.1.12 and later versions, use
DiskCheckpointSpeed
and DiskSyncSize
.
NoOfDiskPagesToDiskDuringRestartTUP
(DEPRECATED)
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | 8K pages/100 milliseconds | 40 | 1 - 4294967039 (0xFFFFFEFF) | N |
This parameter is used in a fashion similar to
NoOfDiskPagesToDiskAfterRestartTUP
and
NoOfDiskPagesToDiskAfterRestartACC
,
only it does so with regard to local checkpoints executed in
the node when a node is restarting. A local checkpoint is
always performed as part of all node restarts. During a node
restart it is possible to write to disk at a higher speed
than at other times, because fewer activities are being
performed in the node.
This parameter covers pages written from data memory.
The default value is 40 (3.2MB per second).
This parameter is deprecated as of MySQL 5.1.6. For MySQL
5.1.12 and later versions, use
DiskCheckpointSpeedInRestart
and DiskSyncSize
.
NoOfDiskPagesToDiskDuringRestartACC
(DEPRECATED)
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | 8K pages/100 milliseconds | 20 | 1 - 4294967039 (0xFFFFFEFF) | N |
Controls the number of index memory pages that can be written to disk during the local checkpoint phase of a node restart.
As with
NoOfDiskPagesToDiskAfterRestartTUP
and
NoOfDiskPagesToDiskAfterRestartACC
,
values for this parameter are expressed in terms of 8KB
pages written per 100 milliseconds (80KB/second).
The default value is 20 (1.6MB per second).
This parameter is deprecated as of MySQL 5.1.6. For MySQL
5.1.12 and later versions, use
DiskCheckpointSpeedInRestart
and DiskSyncSize
.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | milliseconds | 3000 | 10 - 4294967039 (0xFFFFFEFF) | N |
NDB 7.2.0 | milliseconds | 7500 | 10 - 4294967039 (0xFFFFFEFF) | N |
This parameter specifies how long data nodes wait for a response from the arbitrator to an arbitration message. If this is exceeded, the network is assumed to have split.
In MySQL Cluster NDB 7.2.0 and later, the default value is 7500 milliseconds (7.5 seconds). Previously, this was 3000 milliseconds (3 seconds).
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 7.0.7 | enumeration | Default | Default, Disabled, WaitExternal | N |
The Arbitration
parameter, added in MySQL Cluster NDB 7.0.7, enables a
choice of arbitration schemes, corresponding to one of 3
possible values for this parameter:
Default.
This enables arbitration to proceed normally, as
determined by the ArbitrationRank
settings for the management and API nodes. This is the
default value.
Disabled.
Previously, it was possible to disable arbitration
only by setting ArbitrationRank
to
0 on all management and API nodes. Now, you can now
use Arbitration = Disabled
in the
[ndbd default]
section of the
config.ini
file to accomplish
this task. In this case, any
ArbitrationRank
settings are
ignored.
WaitExternal.
The
Arbitration
parameter also makes it possible to configure
arbitration in such a way that the cluster waits until
after the time determined by
ArbitrationTimeout
has passed for an external cluster manager application
to perform arbitration instead of handling arbitration
internally. This can be done by setting
Arbitration = WaitExternal
in the
[ndbd default]
section of the
config.ini
file. For best results
with the WaitExternal
setting, it
is recommended that
ArbitrationTimeout
be 2 times as long as the interval required by the
external cluster manager to perform arbitration.
This parameter should be used only in the [ndbd
default]
section of the cluster configuration
file. The behavior of the cluster is unspecified when
Arbitration
is set
to different values for individual data nodes.
Buffering and logging.
Several [ndbd]
configuration parameters
enable the advanced user to have more control over the
resources used by node processes and to adjust various buffer
sizes at need.
These buffers are used as front ends to the file system when
writing log records to disk. If the node is running in diskless
mode, these parameters can be set to their minimum values
without penalty due to the fact that disk writes are
“faked” by the NDB
storage engine's file system abstraction layer.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | unsigned | 2M | 1M - 4294967039 (0xFFFFFEFF) | N |
The UNDO index buffer, whose size is set by this parameter,
is used during local checkpoints. The
NDB
storage engine uses a
recovery scheme based on checkpoint consistency in
conjunction with an operational REDO log. To produce a
consistent checkpoint without blocking the entire system for
writes, UNDO logging is done while performing the local
checkpoint. UNDO logging is activated on a single table
fragment at a time. This optimization is possible because
tables are stored entirely in main memory.
The UNDO index buffer is used for the updates on the primary key hash index. Inserts and deletes rearrange the hash index; the NDB storage engine writes UNDO log records that map all physical changes to an index page so that they can be undone at system restart. It also logs all active insert operations for each fragment at the start of a local checkpoint.
Reads and updates set lock bits and update a header in the hash index entry. These changes are handled by the page-writing algorithm to ensure that these operations need no UNDO logging.
This buffer is 2MB by default. The minimum value is 1MB,
which is sufficient for most applications. For applications
doing extremely large or numerous inserts and deletes
together with large transactions and large primary keys, it
may be necessary to increase the size of this buffer. If
this buffer is too small, the NDB storage engine issues
internal error code 677 (Index UNDO buffers
overloaded
).
It is not safe to decrease the value of this parameter during a rolling restart.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | unsigned | 16M | 1M - 4294967039 (0xFFFFFEFF) | N |
This parameter sets the size of the UNDO data buffer, which performs a function similar to that of the UNDO index buffer, except the UNDO data buffer is used with regard to data memory rather than index memory. This buffer is used during the local checkpoint phase of a fragment for inserts, deletes, and updates.
Because UNDO log entries tend to grow larger as more operations are logged, this buffer is also larger than its index memory counterpart, with a default value of 16MB.
This amount of memory may be unnecessarily large for some applications. In such cases, it is possible to decrease this size to a minimum of 1MB.
It is rarely necessary to increase the size of this buffer. If there is such a need, it is a good idea to check whether the disks can actually handle the load caused by database update activity. A lack of sufficient disk space cannot be overcome by increasing the size of this buffer.
If this buffer is too small and gets congested, the NDB storage engine issues internal error code 891 (Data UNDO buffers overloaded).
It is not safe to decrease the value of this parameter during a rolling restart.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | bytes | 8M | 1M - 4294967039 (0xFFFFFEFF) | N |
NDB 7.0.4 | bytes | 32M | 1M - 4294967039 (0xFFFFFEFF) | N |
All update activities also need to be logged. The REDO log makes it possible to replay these updates whenever the system is restarted. The NDB recovery algorithm uses a “fuzzy” checkpoint of the data together with the UNDO log, and then applies the REDO log to play back all changes up to the restoration point.
RedoBuffer
sets the size of the buffer in
which the REDO log is written. In MySQL Cluster NDB 6.4.3
and earlier, the default value is 8MB; beginning with MySQL
Cluster NDB 7.0.4, the default is 32MB. The minimum value is
1MB.
If this buffer is too small, the
NDB
storage engine issues error
code 1221 (REDO log buffers
overloaded). For this reason, you should
exercise care if you attempt to decrease the value of
RedoBuffer
as part of an online change in
the cluster's configuration.
ndbmtd allocates a separate buffer for
each LDM thread (see
ThreadConfig
). For
example, with 4 LDM threads, an ndbmtd
data node actually has 4 buffers and allocates
RedoBuffer
bytes to each one, for a total
of 4 * RedoBuffer
bytes.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 7.0.4 | bytes | 8192 | 0 - 64K | S |
Controls the size of the circular buffer used for NDB log events within data nodes.
Added in MySQL Cluster NDB 7.0.4.
Controlling log messages.
In managing the cluster, it is very important to be able to
control the number of log messages sent for various event
types to stdout
. For each event category,
there are 16 possible event levels (numbered 0 through 15).
Setting event reporting for a given event category to level 15
means all event reports in that category are sent to
stdout
; setting it to 0 means that there
will be no event reports made in that category.
By default, only the startup message is sent to
stdout
, with the remaining event reporting
level defaults being set to 0. The reason for this is that these
messages are also sent to the management server's cluster log.
An analogous set of levels can be set for the management client to determine which event levels to record in the cluster log.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 1 | 0 - 15 | N |
The reporting level for events generated during startup of the process.
The default level is 1.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 0 | 0 - 15 | N |
The reporting level for events generated as part of graceful shutdown of a node.
The default level is 0.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 0 | 0 - 15 | N |
The reporting level for statistical events such as number of primary key reads, number of updates, number of inserts, information relating to buffer usage, and so on.
The default level is 0.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | log level | 0 | 0 - 15 | N |
The reporting level for events generated by local and global checkpoints.
The default level is 0.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 0 | 0 - 15 | N |
The reporting level for events generated during node restart.
The default level is 0.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 0 | 0 - 15 | N |
The reporting level for events generated by connections between cluster nodes.
The default level is 0.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 0 | 0 - 15 | N |
The reporting level for events generated by errors and warnings by the cluster as a whole. These errors do not cause any node failure but are still considered worth reporting.
The default level is 0.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | levelr | 0 | 0 - 15 | N |
The reporting level for events generated by congestion. These errors do not cause node failure but are still considered worth reporting.
The default level is 0.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | integer | 0 | 0 - 15 | N |
The reporting level for events generated for information about the general state of the cluster.
The default level is 0.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.1.0 | unsigned | 0 | 0 - 4294967039 (0xFFFFFEFF) | N |
MySQL 5.1.16 | unsigned | 0 | 0 - 4294967039 (0xFFFFFEFF) | N |
This parameter controls how often data node memory usage reports are recorded in the cluster log; it is an integer value representing the number of seconds between reports.
Each data node's data memory and index memory usage is
logged as both a percentage and a number of 32 KB pages of
the DataMemory
and
IndexMemory
,
respectively, set in the config.ini
file. For example, if
DataMemory
is equal
to 100 MB, and a given data node is using 50 MB for data
memory storage, the corresponding line in the cluster log
might look like this:
2006-12-24 01:18:16 [MgmSrvr] INFO -- Node 2: Data usage is 50%(1280 32K pages of total 2560)
MemReportFrequency
is not a required parameter. If used, it can be set for all
cluster data nodes in the [ndbd default]
section of config.ini
, and can also be
set or overridden for individual data nodes in the
corresponding [ndbd]
sections of the
configuration file. The minimum value—which is also
the default value—is 0, in which case memory reports
are logged only when memory usage reaches certain
percentages (80%, 90%, and 100%), as mentioned in the
discussion of statistics events in
Section 17.5.6.2, “MySQL Cluster Log Events”.
This parameter was added in MySQL Cluster 5.1.16 and MySQL Cluster NDB 6.1.0.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.4.0 | seconds | 0 | 0 - 4294967039 (0xFFFFFEFF) | N |
When a data node is started with the
--initial
, it initializes the
redo log file during Start Phase 4 (see
Section 17.5.1, “Summary of MySQL Cluster Start Phases”). When very
large values are set for
NoOfFragmentLogFiles
,
FragmentLogFileSize
,
or both, this initialization can take a long time. Previous
to MySQL Cluster NDB 6.4.0, only the beginning and end of
the redo log file initialization process were logged.
Beginning with this version, it is possible to force reports
on the progress of this process to be logged periodically,
by means of the
StartupStatusReportFrequency
configuration parameter. In this case, progress is reported
in the cluster log, in terms of both the number of files and
the amount of space that have been initialized, as shown
here:
2009-06-20 16:39:23 [MgmSrvr] INFO -- Node 1: Local redo log file initialization status: #Total files: 80, Completed: 60 #Total MBytes: 20480, Completed: 15557 2009-06-20 16:39:23 [MgmSrvr] INFO -- Node 2: Local redo log file initialization status: #Total files: 80, Completed: 60 #Total MBytes: 20480, Completed: 15570
These reports are logged each
StartupStatusReportFrequency
seconds during Start Phase 4. If
StartupStatusReportFrequency
is 0 (the default), then reports are written to the cluster
log only when at the beginning and at the completion of the
redo log file initialization process.
Debugging Parameters.
Beginning with MySQL Cluster NDB 6.3.36, MySQL Cluster NDB
7.0.17, and MySQL Cluster NDB 7.1.6, it is possible to cause
logging of traces for events generated by creating and
dropping tables using
DictTrace
. This
parameter is useful only in debugging NDB kernel code.
DictTrace
takes an
integer value; currently, 0 (default - no logging) and 1
(logging enabled) are the only supported values.
Backup parameters.
The [ndbd]
parameters discussed in this
section define memory buffers set aside for execution of
online backups.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | bytes | 2M | 0 - 4294967039 (0xFFFFFEFF) | N |
NDB 6.3.23 | bytes | 16M | 0 - 4294967039 (0xFFFFFEFF) | N |
NDB 7.0.4 | bytes | 16M | 0 - 4294967039 (0xFFFFFEFF) | N |
In creating a backup, there are two buffers used for sending
data to the disk. The backup data buffer is used to fill in
data recorded by scanning a node's tables. Once this buffer
has been filled to the level specified as
BackupWriteSize
, the
pages are sent to disk. While flushing data to disk, the
backup process can continue filling this buffer until it
runs out of space. When this happens, the backup process
pauses the scan and waits until some disk writes have
completed freeing up memory so that scanning may continue.
In MySQL Cluster NDB 6.4.3 and earlier, the default value is 2MB; in MySQL Cluster NDB 7.0.4 and later, it is 16MB.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | bytes | 2M | 0 - 4294967039 (0xFFFFFEFF) | N |
NDB 6.3.23 | bytes | 16M | 0 - 4294967039 (0xFFFFFEFF) | N |
NDB 7.0.4 | bytes | 16M | 0 - 4294967039 (0xFFFFFEFF) | N |
The backup log buffer fulfills a role similar to that played by the backup data buffer, except that it is used for generating a log of all table writes made during execution of the backup. The same principles apply for writing these pages as with the backup data buffer, except that when there is no more space in the backup log buffer, the backup fails. For that reason, the size of the backup log buffer must be large enough to handle the load caused by write activities while the backup is being made. See Section 17.5.3.3, “Configuration for MySQL Cluster Backups”.
The default value for this parameter should be sufficient for most applications. In fact, it is more likely for a backup failure to be caused by insufficient disk write speed than it is for the backup log buffer to become full. If the disk subsystem is not configured for the write load caused by applications, the cluster is unlikely to be able to perform the desired operations.
It is preferable to configure cluster nodes in such a manner that the processor becomes the bottleneck rather than the disks or the network connections.
In MySQL Cluster NDB 6.4.3 and earlier, the default value is 2MB; in MySQL Cluster NDB 7.0.4 and later, it is 16MB.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | bytes | 4M | 0 - 4294967039 (0xFFFFFEFF) | N |
NDB 7.0.4 | bytes | 32M | 0 - 4294967039 (0xFFFFFEFF) | N |
This parameter is simply the sum of
BackupDataBufferSize
and
BackupLogBufferSize
.
In MySQL Cluster NDB 7.0.4 and later, the default value is 16MB + 16MB = 32MB. (Previously, it was 2MB + 2MB = 4MB.)
If
BackupDataBufferSize
and
BackupLogBufferSize
taken together exceed the default value for
BackupMemory
, then
this parameter must be set explicitly in the
config.ini
file to their sum.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.2.3 | seconds | 0 | 0 - 4294967039 (0xFFFFFEFF) | N |
This parameter controls how often backup status reports are
issued in the management client during a backup, as well as
how often such reports are written to the cluster log
(provided cluster event logging is configured to permit
it—see
Logging and checkpointing).
BackupReportFrequency
represents the time in seconds between backup status
reports.
The default value is 0.
This parameter was added in MySQL Cluster NDB 6.2.3.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | bytes | 32K | 2K - 4294967039 (0xFFFFFEFF) | N |
NDB 7.0.4 | bytes | 256K | 2K - 4294967039 (0xFFFFFEFF) | N |
This parameter specifies the default size of messages written to disk by the backup log and backup data buffers.
In MySQL Cluster 6.4.3 and earlier, the default value for this parameter was 32KB; beginning with MySQL Cluster NDB 7.0.4, it is 256KB.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.0 | bytes | 256K | 2K - 4294967039 (0xFFFFFEFF) | N |
MySQL 5.1.32 | bytes | 1M | 2K - 4294967039 (0xFFFFFEFF) | N |
NDB 7.2.1 | bytes | 1M | 2K - 4294967039 (0xFFFFFEFF) | N |
This parameter specifies the maximum size of messages written to disk by the backup log and backup data buffers.
In MySQL Cluster 6.4.3 and earlier, the default value for this parameter was 256KB; beginning with MySQL Cluster NDB 7.0.4, it is 1MB.
When specifying these parameters, the following relationships must hold true. Otherwise, the data node will be unable to start.
BackupDataBufferSize >= BackupWriteSize +
188KB
BackupLogBufferSize >= BackupWriteSize +
16KB
BackupMaxWriteSize >= BackupWriteSize
The [ndbd]
parameters discussed in this
section are used in scheduling and locking of threads to
specific CPUs on multiprocessor data node hosts. They were
introduced in MySQL Cluster NDB 6.3.4.
To make use of these parameters, the data node process must be run as system root.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.3.4 | CPU ID | 64K | 0 - 64K | N |
Previous to MySQL Cluster NDB 7.0.
This parameter specifies the ID of the CPU assigned to
handle the NDBCLUSTER
execution thread. The value of this parameter is an
integer in the range 0 to 65535 (inclusive). The default
is 65535.
MySQL Cluster NDB 7.0 and later (beginning with MySQL Cluster NDB 6.4.0).
When used with ndbd, this parameter
(now a string) specifies the ID of the CPU assigned to
handle the NDBCLUSTER
execution thread.
When used with ndbmtd, the value of this
parameter is a comma-separated list of CPU IDs assigned to
handle execution threads. Each CPU ID in the list should be
an integer in the range 0 to 65535 (inclusive). The number
of IDs specified should match the number of execution
threads determined by
MaxNoOfExecutionThreads
.
However, there is no guarantee that threads are assigned to
CPUs in any given order when using this parameter; beginning
with in MySQL Cluster NDB 7.0.31 and MySQL Cluster NDB
7.1.20, you can obtain more finely-grained control of this
type using
ThreadConfig
.
LockExecuteThreadToCPU
has no default value.
Prior to MySQL Cluster NDB 7.0.18 and MySQL Cluster NDB 7.1.7, the effective maximum value recognized by this parameter as a valid CPU ID was 255; using a greater value caused thread locking to be disabled. (Bug #56185)
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.3.4 | CPU ID | 64K | 0 - 64K | N |
NDB 6.4.0 | CPU ID | [none] | 0 - 64K | N |
This parameter specifies the ID of the CPU assigned to
handle NDBCLUSTER
maintenance
threads.
The value of this parameter is an integer in the range 0 to 65535 (inclusive). This parameter was added in MySQL Cluster NDB 6.3.4. Prior to MySQL Cluster NDB 6.4.0, the default is 65535; in MySQL Cluster NDB 7.0 and later MySQL Cluster release series, there is no default value.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.3.4 | boolean | false | true, false | N |
Setting this parameter to 1 enables real-time scheduling of data node threads.
Prior to MySQL Cluster NDB 7.0.40 and MySQL Cluster NDB 7.1.29, this parameter did not work correctly with data nodes running ndbmtd. (Bug #16961971)
The default is 0 (scheduling disabled).
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.3.4 | µsec | 50 | 0 - 11000 | N |
This parameter specifies the time in microseconds for threads to be executed in the scheduler before being sent. Setting it to 0 minimizes the response time; to achieve higher throughput, you can increase the value at the expense of longer response times.
The default is 50 μsec, which our testing shows to increase throughput slightly in high-load cases without materially delaying requests.
This parameter was added in MySQL Cluster NDB 6.3.4.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.3.4 | µsec | 0 | 0 - 500 | N |
This parameter specifies the time in microseconds for threads to be executed in the scheduler before sleeping.
The default value is 0.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.3.30 | numeric | 0 | 0 - 128 | S |
NDB 7.0.11 | numeric | 0 | 0 - 128 | S |
This parameter determines the number of threads to create
when rebuilding ordered indexes during a system or node
start, as well as when running
ndb_restore
--rebuild-indexes
. It is
supported only when there is more than one fragment for the
table per data node (for example, when the
MAX_ROWS
option has been used with
CREATE TABLE
).
Setting this parameter to 0 (the default) disables multi-threaded building of ordered indexes.
This parameter was added in MySQL Cluster NDB 6.3.30 and
MySQL Cluster NDB 7.0.11. Prior to MySQL Cluster NDB 7.0.16
and MySQL Cluster NDB 7.1.5, it was supported only when
using ndbd; in these and later MySQL
Cluster releases,
BuildIndexThreads
is
also supported for data nodes running
ndbmtd (see Bug #54521).
You can enable multi-threaded builds during data node
initial restarts by setting the
TwoPassInitialNodeRestartCopy
data node configuration parameter to
TRUE
.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 7.1.11 | boolean | false | true, false | N |
In MySQL Cluster NDB 7.1.11 and later, multi-threaded
building of ordered indexes can be enabled for initial
restarts of data nodes by setting this configuration
parameter to TRUE
, which enables two-pass
copying of data during initial node restarts.
You must also set
BuildIndexThreads
to
a nonzero value.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 7.0.20 | boolean | 1 | ... | N |
NDB 7.1.9 | boolean | 1 | ... | N |
NDB
is extremely sensitive to
Non-Uniform Memory Access settings and multi-CPU systems due
to timeouts that it can cause. Due to this fact, and because
most MySQL Cluster users do not employ
numactl, support for NUMA is ignored by
default by ndbd when running on a Linux
system. If your Linux system provides NUMA support and you
wish for data node memory to be subject to NUMA control, you
can set this parameter equal to 0.
The Numa
configuration parameter is supported only on Linux systems
where libnuma.so
is installed.
Multi-Threading Configuration Parameters (ndbmtd).
ndbmtd runs by default as a single-threaded
process and must be configured to use multiple threads, using
either of two methods, both of which require setting
configuration parameters in the
config.ini
file. The first method is
simply to set an appropriate value for the
MaxNoOfExecutionThreads
configuration parameter. In MySQL Cluster NDB 7.1.17 and
later, a second method is also supported, whereby it is
possible to set up more complex rules for
ndbmtd multi-threading using
ThreadConfig
. The
next few paragraphs provide information about these parameters
and their use with multi-threaded data nodes.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.4.0 | integer | [none] | 2 - 8 | IS |
NDB 7.0.4 | integer | 2 | 2 - 8 | IS |
This parameter controls the number of execution threads used
by ndbmtd, up to a maximum of 8 threads,
4 of which can be LDM (LQH) threads. Although this parameter
is set in the [ndbd]
or [ndbd
default]
sections of the
config.ini
file, it is exclusive to
ndbmtd and does not apply to
ndbd.
Setting MaxNoOfExecutionThreads
sets the
number of threads by type as determined in the following
table:
MaxNoOfExecutionThreads Value | LDM Threads | TC Threads | Send Threads | Receive Threads |
---|---|---|---|---|
0 .. 3 | 1 | 1 | 0 | 1 |
4 .. 6 | 2 | 1 | 0 | 1 |
7 .. 8 | 4 | 1 | 0 | 1 |
There is always one SUMA (replication) thread.
The thread types are described later in this section (see
ThreadConfig
).
Setting this parameter outside the permitted range of values
causes the management server to abort on startup with the
error Error line
number
: Illegal value
value
for parameter
MaxNoOfExecutionThreads.
For MaxNoOfExecutionThreads
, a value of 0
or 1 is rounded up internally by
NDB
to 2, so that 2 is
considered this parameter's default and minimum value.
MaxNoOfExecutionThreads
is generally
intended to be set equal to the number of CPU threads
available, and to allocate a number of threads of each type
suitable to typical workloads. It does not assign particular
threads to specified CPUs. For cases where it is desirable
to vary from the settings provided, or to bind threads to
CPUs, you should use
ThreadConfig
instead, which allows you to allocate each thread directly
to a desired type, CPU, or both.
The multi-threaded data node process always spawns at least 4 threads, listed here:
1 local query handler (LDM) thread
1 transaction coordinator (TC) thread
1 receive thread
1 subscription manager (SUMA or replication) thread
Changing the number of LDM threads always requires a system
restart, whether it is changed using this parameter or
ThreadConfig
. If
the cluster's
IndexMemory
usage is
greater than 50%, changing this requires an initial restart
of the cluster. (A maximum of 30-35%
IndexMemory
usage is recommended in such
cases.) Otherwise, resource usage and LDM thread allocation
cannot be balanced between nodes, which can result in
underutilized and overutilized LDM threads, and ultimately
data node failures.
In MySQL Cluster NDB 7.1 and earlier, it is not possible to cause ndbmtd to use more than 1 TC thread. (This is possible in MySQL Cluster NDB 7.2.)
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 7.1.17 | string | '' | ... | IS |
NDB 7.2.3 | string | '' | ... | IS |
This parameter is used with ndbmtd to assign threads of different types to different CPUs. Its value is a string whose format has the following syntax:
ThreadConfig :=entry
[,entry
[,...]]entry
:=type
={param
[,param
[,...]]}type
:= ldm | main | recv | rep | ioparam
:= count=number
| cpubind=cpu_list
The curly braces
({
...}
) surrounding
the list of parameters is required, even if there is only
one parameter in the list.
A param
(parameter) specifies the
number of threads of the given type
(count
), the CPUs to which the threads of
the given type are to be bound (cpubind
),
or both.
The type
attribute represents an
NDB thread type. The thread types supported in MySQL Cluster
NDB 7.1 and the range of permitted count
values for each are provided in the following list:
ldm
: Local query handler
(DBLQH
kernel block) that handles
data. The more LDM threads that are used, the more
highly partitioned the data becomes. Each LDM thread
maintains its own sets of data and index partitions, as
well as its own redo log. In MySQL Cluster NDB 7.1, the
maximum is 4 such threads.
Changing the number of LDM threads requires a system
restart to be effective and safe for cluster
operations. (This is also true when this is done using
MaxNoOfExecutionThreads
.)
If IndexMemory
usage is in excess of 50%, an initial restart of the
cluster is required; a maximum of 30-35%
IndexMemory
usage is recommended in
such cases. Otherwise, IndexMemory
and DataMemory
usage as well as the allocation of LDM threads cannot
be balanced between nodes, which can ultimately lead
to data node failures.
tc
: Transaction coordinator thread
(DBTC
kernel block) containing the
state of an ongoing transaction. In MySQL Cluster NDB
7.1, there can be only 1 TC thread. (In MySQL Cluster
NDB 7.2, this number is configurable.)
Range: 1 only.
main
: Data dictionary and transaction
coordinator (DBDIH
and
DBTC
kernel blocks), providing schema
management. This is always handled by a single dedicated
thread.
Range: 1 only.
recv
: Receive thread
(CMVMI
kernel block). Each receive
thread handles one or more sockets for communicating
with other nodes in a MySQL Cluster, with one socket per
node. Previously, this was limited to a single thread,
but MySQL Cluster 7.2 implements multiple receive
threads (up to 8).
Range: 1 only.
rep
: Replication thread
(SUMA
kernel block). Asynchronous
replication operations are always handled by a
single.dedicated thread.
Range: 1 only.
io
: File system and other
miscellaneous operations. These are not demanding tasks,
and are always handled as a group by a single, dedicated
I/O thread.
Range: 1 only.
Simple examples:
# Example 1. ThreadConfig=ldm={count=2,cpubind=1,2},main={cpubind=12},rep={cpubind=11} # Example 2. Threadconfig=main={cpubind=0},ldm={count=4,cpubind=1,2,5,6},io={cpubind=3}
It is usually desirable when configuring thread usage for a data
node host to reserve one or more CPUs for operating system and
other tasks. Thus, for a host machine with 8 CPUs, you might
want to use 8 CPU threads bound to 7 CPUs (leaving one for
operating system and other functions), with 4 LDM threads, 1 TC
thread, 1 receive thread, and 1 thread each for schema
management, asynchronous replication, and I/O operations. (This
is almost the same distribution of threads used when
MaxNoOfExecutionThreads
is set
equal to 8.) The following ThreadConfig
setting performs these assignments, additionally binding all of
these threads to specific CPUs:
ThreadConfig=ldm{count=4,cpubind=1,2,3,4},main={cpubind=5}, \ io={cpubind=5},rep={cpubind=6},tc{cpubind=7},recv={cpubind=8}
It should be possible in most cases to bind the main (schema management) thread and the I/O thread to the same CPU, as we have done in the example just shown.
In order to take advantage of the enhanced stability that the
use of ThreadConfig
offers, it is necessary
to insure that CPUs are isolated, and that they not subject to
interrupts, or to being scheduled for other tasks by the
operating system. On many Linux systems, you can do this by
setting IRQBALANCE_BANNED_CPUS
in
/etc/sysconfig/irqbalance
to
0xFFFFF0
, and by using the
isolcpus
boot option in
grub.conf
. For specific information, see
your operating system or platform documentation.
In MySQL Cluster NDB 7.1 and earlier, it is not possible to cause ndbmtd to use more than 1 TC thread; this capability is introduced in MySQL Cluster NDB 7.2.
Disk Data Configuration Parameters. Configuration parameters affecting Disk Data behavior include the following:
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
MySQL 5.1.6 | bytes | 64M | 4M - 1T | N |
This determines the amount of space used for caching pages
on disk, and is set in the [ndbd]
or
[ndbd default]
section of the
config.ini
file. It is measured in
bytes. Each page takes up 32 KB. This means that Cluster
Disk Data storage always uses N
*
32 KB memory where N
is some
nonnegative integer.
The default value for this parameter is
64M
(2000 pages of 32 KB each).
This parameter was added in MySQL 5.1.6.
Beginning with MySQL Cluster NDB 7.1.9, you can query the
ndbinfo.diskpagebuffer
table to help determine whether the value for this parameter
should be increased to minimize unnecessary disk seeks. See
Section 17.5.10.8, “The ndbinfo diskpagebuffer Table”, for
more information.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 7.2.0 | bytes | 128M | 0 - 64T | N |
MySQL 5.1.6 | bytes | 20M | 0 - 64T | N |
This parameter determines the amount of memory that is used
for log buffers, disk operations (such as page requests and
wait queues), and metadata for tablespaces, log file groups,
UNDO
files, and data files. The shared
global memory pool also provides memory used for satisfying
the memory requirements of the
INITIAL_SIZE
and
UNDO_BUFFER_SIZE
options used with
CREATE LOGFILE GROUP
and
ALTER LOGFILE GROUP
statements, including any default value implied for these
options by the setting of the
InitialLogFileGroup
data node configuration parameter.
SharedGlobalMemory
can be set in the
[ndbd]
or [ndbd
default]
section of the
config.ini
configuration file, and is
measured in bytes.
As of MySQL Cluster NDB 7.2.0, the default value is
128M
. (Previously, this was
20M
.)
This parameter was added in MySQL 5.1.6.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.4.3 | threads | 8 | 0 - 4294967039 (0xFFFFFEFF) | N |
NDB 7.0.7 | threads | 2 | 0 - 4294967039 (0xFFFFFEFF) | N |
NDB 7.1.0 | threads | 2 | 0 - 4294967039 (0xFFFFFEFF) | N |
This parameter determines the number of unbound threads used
for Disk Data file access. Before
DiskIOThreadPool
was
introduced, exactly one thread was spawned for each Disk
Data file, which could lead to performance issues,
particularly when using very large data files. With
DiskIOThreadPool
,
you can—for example—access a single large data
file using several threads working in parallel.
This parameter applies to Disk Data I/O threads only.
The optimum value for this parameter depends on your hardware and configuration, and includes these factors:
Physical distribution of Disk Data files.
You can obtain better performance by placing data
files, undo log files, and the data node file system
on separate physical disks. If you do this with some
or all of these sets of files, then you can set
DiskIOThreadPool
higher to enable separate threads to handle the files
on each disk.
Disk performance and types.
The number of threads that can be accommodated for
Disk Data file handling is also dependent on the speed
and throughput of the disks. Faster disks and higher
throughput allow for more disk I/O threads. Our test
results indicate that solid-state disk drives can
handle many more disk I/O threads than conventional
disks, and thus higher values for
DiskIOThreadPool
.
This parameter was added in MySQL Cluster NDB 6.4.0.
Previous to MySQL Cluster NDB 6.4.3, it was named
IOThreadPool
. Previous to MySQL Cluster
NDB 7.0.7, the default value was 8. Beginning with MySQL
Cluster NDB 7.0.7 and MySQL Cluster NDB 7.1.0, the default
is 2.
Disk Data file system parameters. The parameters in the following list were added in MySQL Cluster NDB 6.2.17, 6.3.22, and 6.4.3 to make it possible to place MySQL Cluster Disk Data files in specific directories without the need for using symbolic links.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.3.22 | filename | [see text] | ... | IN |
NDB 6.2.17 | filename | [see text] | ... | IN |
NDB 6.4.3 | filename | [see text] | ... | IN |
If this parameter is specified, then MySQL Cluster Disk
Data data files and undo log files are placed in the
indicated directory. This can be overridden for data
files, undo log files, or both, by specifying values for
FileSystemPathDataFiles
,
FileSystemPathUndoFiles
,
or both, as explained for these parameters. It can also
be overridden for data files by specifying a path in the
ADD DATAFILE
clause of a
CREATE TABLESPACE
or
ALTER TABLESPACE
statement, and for undo log files by specifying a path
in the ADD UNDOFILE
clause of a
CREATE LOGFILE GROUP
or
ALTER LOGFILE GROUP
statement. If
FileSystemPathDD
is not specified, then
FileSystemPath
is used.
If a
FileSystemPathDD
directory is specified for a given data node (including
the case where the parameter is specified in the
[ndbd default]
section of the
config.ini
file), then starting
that data node with --initial
causes
all files in the directory to be deleted.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.3.22 | filename | [see text] | ... | IN |
NDB 6.2.17 | filename | [see text] | ... | IN |
NDB 6.4.3 | filename | [see text] | ... | IN |
If this parameter is specified, then MySQL Cluster Disk
Data data files are placed in the indicated directory.
This overrides any value set for
FileSystemPathDD
.
This parameter can be overridden for a given data file
by specifying a path in the ADD
DATAFILE
clause of a
CREATE TABLESPACE
or
ALTER TABLESPACE
statement used to create that data file. If
FileSystemPathDataFiles
is not specified, then
FileSystemPathDD
is used (or
FileSystemPath
,
if
FileSystemPathDD
has also not been set).
If a
FileSystemPathDataFiles
directory is specified for a given data node (including
the case where the parameter is specified in the
[ndbd default]
section of the
config.ini
file), then starting
that data node with --initial
causes
all files in the directory to be deleted.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.3.22 | filename | [see text] | ... | IN |
NDB 6.2.17 | filename | [see text] | ... | IN |
NDB 6.4.3 | filename | [see text] | ... | IN |
If this parameter is specified, then MySQL Cluster Disk
Data undo log files are placed in the indicated
directory. This overrides any value set for
FileSystemPathDD
.
This parameter can be overridden for a given data file
by specifying a path in the ADD UNDO
clause of a CREATE LOGFILE
GROUP
or CREATE LOGFILE
GROUP
statement used to create that data file.
If
FileSystemPathUndoFiles
is not specified, then
FileSystemPathDD
is used (or
FileSystemPath
,
if
FileSystemPathDD
has also not been set).
If a
FileSystemPathUndoFiles
directory is specified for a given data node (including
the case where the parameter is specified in the
[ndbd default]
section of the
config.ini
file), then starting
that data node with --initial
causes
all files in the directory to be deleted.
For more information, see Section 17.5.12.1, “MySQL Cluster Disk Data Objects”.
Disk Data object creation parameters. The next two parameters enable you—when starting the cluster for the first time—to cause a Disk Data log file group, tablespace, or both, to be created without the use of SQL statements.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.3.22 | string | [see text] | ... | S |
NDB 6.2.17 | string | [see text] | ... | S |
NDB 6.4.3 | string | [see text] | ... | S |
This parameter can be used to specify a log file group
that is created when performing an initial start of the
cluster.
InitialLogFileGroup
is specified as shown here:
InitialLogFileGroup = [name=name
;] [undo_buffer_size=size
;]file-specification-list
file-specification-list
:file-specification
[;file-specification
[; ...]]file-specification
:filename
:size
The name
of the log file group is
optional and defaults to DEFAULT-LG
.
The undo_buffer_size
is also
optional; if omitted, it defaults to
64M
. Each
file-specification
corresponds to an undo log file, and at least one must
be specified in the
file-specification-list
. Undo
log files are placed according to any values that have
been set for
FileSystemPath
,
FileSystemPathDD
,
and
FileSystemPathUndoFiles
,
just as if they had been created as the result of a
CREATE LOGFILE GROUP
or
ALTER LOGFILE GROUP
statement.
Consider the following:
InitialLogFileGroup = name=LG1; undo_buffer_size=128M; undo1.log:250M; undo2.log:150M
This is equivalent to the following SQL statements:
CREATE LOGFILE GROUP LG1 ADD UNDOFILE 'undo1.log' INITIAL_SIZE 250M UNDO_BUFFER_SIZE 128M ENGINE NDBCLUSTER; ALTER LOGFILE GROUP LG1 ADD UNDOFILE 'undo2.log' INITIAL_SIZE 150M ENGINE NDBCLUSTER;
This logfile group is created when the data nodes are
started with --initial
.
Resources for the initial log file group are taken from
the global memory pool whose size is determined by the
value of the
SharedGlobalMemory
data node configuration parameter; if this parameter is
set too low and the values set in
InitialLogFileGroup
for the logfile
group's initial size or undo buffer size are too
high, the cluster may fail to create the default log
file group when starting, or fail to start altogether.
This parameter, if used, should always be set in the
[ndbd default]
section of the
config.ini
file. The behavior of a
MySQL Cluster when different values are set on different
data nodes is not defined.
Effective Version | Type/Units | Default | Range/Values | Restart Type |
---|---|---|---|---|
NDB 6.3.22 | string | [see text] | ... | S |
NDB 6.2.17 | string | [see text] | ... | S |
NDB 6.4.3 | string | [see text] | ... | S |
This parameter can be used to specify a MySQL Cluster
Disk Data tablespace that is created when performing an
initial start of the cluster.
InitialTablespace
is specified as shown here:
InitialTablespace = [name=name
;] [extent_size=size
;]file-specification-list
The name
of the tablespace is
optional and defaults to DEFAULT-TS
.
The extent_size
is also optional; it
defaults to 1M
. The
file-specification-list
uses
the same syntax as shown with the
InitialLogfileGroup
parameter, the only difference being that each
file-specification
used with
InitialTablespace
corresponds to a data file. At least one must be