Go to the first, previous, next, last section, table of contents.

4 MySQL Database Administration

4.1 Configuring MySQL

4.1.1 mysqld Command-line Options

mysqld accepts the following command-line options:

--ansi

Use ANSI SQL syntax instead of MySQL syntax. See section 1.4.3 Running MySQL in ANSI Mode.

-b, --basedir=path

Path to installation directory. All paths are usually resolved relative to this.

--big-tables

Allow big result sets by saving all temporary sets on file. It solves most 'table full' errors, but also slows down the queries where in-memory tables would suffice. Since Version 3.23.2, MySQL is able to solve it automatically by using memory for small temporary tables and switching to disk tables where necessary.

--bind-address=IP

IP address to bind to.

--character-sets-dir=path

Directory where character sets are. See section 4.6.1 The Character Set Used for Data and Sorting.

--chroot=path

Chroot mysqld daemon during startup. Recommended security measure. It will somewhat limit LOAD DATA INFILE and SELECT ... INTO OUTFILE though.

--core-file

Write a core file if mysqld dies. For some systems you must also specify --core-file-size to safe_mysqld. See section 4.7.2 safe_mysqld, the wrapper around mysqld.

-h, --datadir=path

Path to the database root.

--default-character-set=charset

Set the default character set. See section 4.6.1 The Character Set Used for Data and Sorting.

--default-table-type=type

Set the default table type for tables. See section 7 MySQL Table Types.

--debug[...]=

If MySQL is configured with --with-debug, you can use this option to get a trace file of what mysqld is doing. See section G.1.2 Creating trace files.

--delay-key-write-for-all-tables

Don't flush key buffers between writes for any MyISAM table. See section 5.5.2 Tuning Server Parameters.

--enable-locking

Enable system locking. Note that if you use this option on a system which a not fully working lockd() (as on Linux) you will easily get mysqld to deadlock.

-T, --exit-info

This is a bit mask of different flags one can use for debugging the mysqld server; One should not use this option if one doesn't know exactly what it does!

--flush

Flush all changes to disk after each SQL command. Normally MySQL only does a write of all changes to disk after each SQL command and lets the operating system handle the syncing to disk. See section A.4.1 What To Do If MySQL Keeps Crashing.

-?, --help

Display short help and exit.

--init-file=file

Read SQL commands from this file at startup.

-L, --language=...

Client error messages in given language. May be given as a full path. See section 4.6.2 Non-English Error Messages.

-l, --log[=file]

Log connections and queries to file. See section 4.9.2 The General Query Log.

--log-isam[=file]

Log all ISAM/MyISAM changes to file (only used when debugging ISAM/MyISAM).

--log-slow-queries[=file]

Log all queries that have taken more than long_query_time seconds to execute to file. See section 4.9.5 The Slow Query Log.

--log-update[=file]

Log updates to file.# where # is a unique number if not given. See section 4.9.3 The Update Log.

--log-long-format

Log some extra information to update log. If you are using --log-slow-queries then queries that are not using indexes are logged to the slow query log.

--low-priority-updates

Table-modifying operations (INSERT/DELETE/UPDATE) will have lower priority than selects. It can also be done via {INSERT | REPLACE | UPDATE | DELETE} LOW_PRIORITY ... to lower the priority of only one query, or by SET OPTION SQL_LOW_PRIORITY_UPDATES=1 to change the priority in one thread. See section 5.3.2 Table Locking Issues.

--memlock

Lock the mysqld process in memory. This works only if your system supports the mlockall() system call (like Solaris). This may help if you have a problem where the operating system is causing mysqld to swap on disk.

--myisam-recover [=option[,option...]]] where option is any combination

of DEFAULT, BACKUP, FORCE or QUICK. You can also set this explicitely to "" if you want to disable this option. If this option is used, mysqld will on open check if the table is marked as crashed or if if the table wasn't closed properly. (The last option only works if you are running with --skip-locking). If this is the case mysqld will run check on the table. If the table was corrupted, mysqld will attempt to repair it. The following options affects how the repair works.

DEFAULT	The same as not giving any option to --myisam-recover.
BACKUP	If the data table was changed during recover, save a backup of the `table_name.MYD' data file as `table_name-datetime.BAK'.
FORCE	Run recover even if we will loose more than one row from the .MYD file.
QUICK	Don't check the rows in the table if there isn't any delete blocks.

Before a table is automatically repaired, MySQL will add a note about this in the error log. If you want to be able to recover from most things without user intervention, you should use the options BACKUP,FORCE. This will force a repair of a table even if some rows would be deleted, but it will keep the old data file as a backup so that you can later examine what happened.

--pid-file=path

Path to pid file used by safe_mysqld.

-P, --port=...

Port number to listen for TCP/IP connections.

-o, --old-protocol

Use the 3.20 protocol for compatibility with some very old clients. See section 2.5.4 Upgrading from Version 3.20 to Version 3.21.

--one-thread

Only use one thread (for debugging under Linux). See section G.1 Debugging a MySQL server.

-O, --set-variable var=option

Give a variable a value. --help lists variables. You can find a full description for all variables in the SHOW VARIABLES section in this manual. See section 4.5.5.4 SHOW VARIABLES. The tuning server parameters section includes information of how to optimize these. See section 5.5.2 Tuning Server Parameters.

--safe-mode

Skip some optimize stages. Implies --skip-delay-key-write.

--safe-show-database

Don't show databases for which the user doesn't have any privileges.

--safe-user-create

If this is enabled, a user can't create new users with the GRANT command, if the user doesn't have INSERT privilege to the mysql.user table or any column in this table.

--skip-concurrent-insert

Turn off the ability to select and insert at the same time on MyISAM tables. (This is only to be used if you think you have found a bug in this feature).

--skip-delay-key-write

Ignore the delay_key_write option for all tables. See section 5.5.2 Tuning Server Parameters.

--skip-grant-tables

This option causes the server not to use the privilege system at all. This gives everyone full access to all databases! (You can tell a running server to start using the grant tables again by executing mysqladmin flush-privileges or mysqladmin reload.)

--skip-host-cache

Never use host name cache for faster name-ip resolution, but query DNS server on every connect instead. See section 5.5.5 How MySQL uses DNS.

--skip-locking

Don't use system locking. To use isamchk or myisamchk you must shut down the server. See section 1.1.7 How Stable Is MySQL?. Note that in MySQL Version 3.23 you can use REPAIR and CHECK to repair/check MyISAM tables.

--skip-name-resolve

Hostnames are not resolved. All Host column values in the grant tables must be IP numbers or localhost. See section 5.5.5 How MySQL uses DNS.

--skip-networking

Don't listen for TCP/IP connections at all. All interaction with mysqld must be made via Unix sockets. This option is highly recommended for systems where only local requests are allowed. See section 5.5.5 How MySQL uses DNS.

--skip-new

Don't use new, possible wrong routines. Implies --skip-delay-key-write. This will also set default table type to ISAM. See section 7.3 ISAM Tables.

--skip-symlink

Don't delete or rename files that a symlinked file in the data directory points to.

--skip-safemalloc

If MySQL is configured with --with-debug=full, all programs will check the memory for overruns for every memory allocation and memory freeing. As this checking is very slow, you can avoid this, when you don't need memory checking, by using this option.

--skip-show-database

Don't allow 'SHOW DATABASE' commands, unless the user has process privilege.

--skip-stack-trace

Don't write stack traces. This option is useful when you are running mysqld under a debugger. See section G.1 Debugging a MySQL server.

--skip-thread-priority

Disable using thread priorities for faster response time.

--socket=path

Socket file to use for local connections instead of default /tmp/mysql.sock.

--sql-mode=option[,option[,option...]]

Option can be any combination of: REAL_AS_FLOAT, PIPES_AS_CONCAT, ANSI_QUOTES, IGNORE_SPACE, SERIALIZE, ONLY_FULL_GROUP_BY. It can also be empty ("") if you want to reset this. By specifying all of the above options is same as using --ansi. With this option one can turn on only needed SQL modes. See section 1.4.3 Running MySQL in ANSI Mode.

transaction-isolation= { READ-UNCOMMITTED | READ-COMMITTED | REPEATABLE-READ | SERIALIZABLE }

Sets the default transaction isolation level. See section 6.7.3 SET TRANSACTION Syntax.

-t, --tmpdir=path

Path for temporary files. It may be useful if your default /tmp directory resides on a partition too small to hold temporary tables.

-u, --user=user_name

Run mysqld daemon as user user_name. This option is mandatory when starting mysqld as root.

-V, --version

Output version information and exit.

-W, --warnings

Print out warnings like Aborted connection... to the .err file. See section A.2.9 Communication Errors / Aborted Connection.

4.1.2 my.cnf Option Files

MySQL can, since Version 3.22, read default startup options for the server and for clients from option files.

MySQL reads default options from the following files on Unix:

Filename	Purpose
/etc/my.cnf	Global options
DATADIR/my.cnf	Server-specific options
defaults-extra-file	The file specified with --defaults-extra-file=#
~/.my.cnf	User-specific options

DATADIR is the MySQL data directory (typically `/usr/local/mysql/data' for a binary installation or `/usr/local/var' for a source installation). Note that this is the directory that was specified at configuration time, not the one specified with --datadir when mysqld starts up! (--datadir has no effect on where the server looks for option files, because it looks for them before it processes any command-line arguments.)

MySQL reads default options from the following files on Windows:

Note that on Windows, you should specify all paths with / instead of \. If you use \, you need to specify this twice, as \ is the escape character in MySQL.

MySQL tries to read option files in the order listed above. If multiple option files exist, an option specified in a file read later takes precedence over the same option specified in a file read earlier. Options specified on the command line take precedence over options specified in any option file. Some options can be specified using environment variables. Options specified on the command line or in option files take precedence over environment variable values. See section H Environment Variables.

The following programs support option files: mysql, mysqladmin, mysqld, mysqldump, mysqlimport, mysql.server, myisamchk, and myisampack.

You can use option files to specify any long option that a program supports! Run the program with --help to get a list of available options.

An option file can contain lines of the following forms:

#comment

Comment lines start with `#' or `;'. Empty lines are ignored.

[group]

group is the name of the program or group for which you want to set options. After a group line, any option or set-variable lines apply to the named group until the end of the option file or another group line is given.

option

This is equivalent to --option on the command line.

option=value

This is equivalent to --option=value on the command line.

set-variable = variable=value

This is equivalent to --set-variable variable=value on the command line. This syntax must be used to set a mysqld variable.

The client group allows you to specify options that apply to all MySQL clients (not mysqld). This is the perfect group to use to specify the password you use to connect to the server. (But make sure the option file is readable and writable only by yourself.)

Note that for options and values, all leading and trailing blanks are automatically deleted. You may use the escape sequences `\b', `\t', `\n', `\r', `\\', and `\s' in your value string (`\s' == blank).

Here is a typical global option file:

[client]
port=3306
socket=/tmp/mysql.sock

[mysqld]
port=3306
socket=/tmp/mysql.sock
set-variable = key_buffer_size=16M
set-variable = max_allowed_packet=1M

[mysqldump]
quick

Here is typical user option file:

[client]
# The following password will be sent to all standard MySQL clients
password=my_password

[mysql]
no-auto-rehash
set-variable = connect_timeout=2

[mysqlhotcopy]
interactive-timeout

If you have a source distribution, you will find sample configuration files named `my-xxxx.cnf' in the `support-files' directory. If you have a binary distribution, look in the `DIR/support-files' directory, where DIR is the pathname to the MySQL installation directory (typically `/usr/local/mysql'). Currently there are sample configuration files for small, medium, large, and very large systems. You can copy `my-xxxx.cnf' to your home directory (rename the copy to `.my.cnf') to experiment with this.

All MySQL clients that support option files support the following options:

Note that the above options must be first on the command line to work! --print-defaults may however be used directly after the --defaults-xxx-file commands.

Note for developers: Option file handling is implemented simply by processing all matching options (that is, options in the appropriate group) before any command-line arguments. This works nicely for programs that use the last instance of an option that is specified multiple times. If you have an old program that handles multiply-specified options this way but doesn't read option files, you need add only two lines to give it that capability. Check the source code of any of the standard MySQL clients to see how to do this.

In shell scripts you can use the `my_print_defaults' command to parse the config files:

shell> my_print_defaults client mysql
--port=3306
--socket=/tmp/mysql.sock
--no-auto-rehash

The above output contains all options for the groups 'client' and 'mysql'.

4.1.3 Installing Many Servers on the Same Machine

In some cases you may want to have many different mysqld daemons (servers) running on the same machine. You may for example want to run a new version of MySQL for testing together with an old version that is in production. Another case is when you want to give different users access to different mysqld servers that they manage themselves.

One way to get a new server running is by starting it with a different socket and port as follows:

shell> MYSQL_UNIX_PORT=/tmp/mysqld-new.sock
shell> MYSQL_TCP_PORT=3307
shell> export MYSQL_UNIX_PORT MYSQL_TCP_PORT
shell> scripts/mysql_install_db
shell> bin/safe_mysqld &

The environment variables appendix includes a list of other environment variables you can use to affect mysqld. See section H Environment Variables.

The above is the quick and dirty way that one commonly uses for testing. The nice thing with this is that all connections you do in the above shell will automatically be directed to the new running server!

If you need to do this more permanently, you should create an option file for each server. See section 4.1.2 my.cnf Option Files. In your startup script that is executed at boot time (mysql.server?) you should specify for both servers:

safe_mysqld --default-file=path-to-option-file

At least the following options should be different per server:

port=#

socket=path

pid-file=path

The following options should be different, if they are used:

log=path

log-bin=path

log-update=path

log-isam=path

bdb-logdir=path

If you want more performance, you can also specify the following differently:

tmpdir=path

bdb-tmpdir=path

See section 4.1.1 mysqld Command-line Options.

If you are installing binary MySQL versions (.tar files) and start them with ./bin/safe_mysqld then in most cases the only option you need to add/change is the socket and port argument to safe_mysqld.

4.1.4 Running Multiple MySQL Servers on the Same Machine

There are circumstances when you might want to run multiple servers on the same machine. For example, you might want to test a new MySQL release while leaving your existing production setup undisturbed. Or you might be an Internet service provider that wants to provide independent MySQL installations for different customers.

If you want to run multiple servers, the easiest way is to compile the servers with different TCP/IP ports and socket files so they are not both listening to the same TCP/IP port or socket file. See section 4.7.3 mysqld_multi, program for managing multiple MySQL servers.

Assume an existing server is configured for the default port number and socket file. Then configure the new server with a configure command something like this:

shell> ./configure  --with-tcp-port=port_number \
             --with-unix-socket-path=file_name \
             --prefix=/usr/local/mysql-3.22.9

Here port_number and file_name should be different than the default port number and socket file pathname, and the --prefix value should specify an installation directory different than the one under which the existing MySQL installation is located.

You can check the socket used by any currently executing MySQL server with this command:

shell> mysqladmin -h hostname --port=port_number variables

Note that if you specify ``localhost'' as a hostname, mysqladmin will default to using Unix sockets instead of TCP/IP.

If you have a MySQL server running on the port you used, you will get a list of some of the most important configurable variables in MySQL, including the socket name.

You don't have to recompile a new MySQL server just to start with a different port and socket. You can change the port and socket to be used by specifying them at run time as options to safe_mysqld:

shell> /path/to/safe_mysqld --socket=file_name --port=port_number

mysqld_multi can also take safe_mysqld (or mysqld) as an argument and pass the options from a configuration file to safe_mysqld and further to mysqld.

If you run the new server on the same database directory as another server with logging enabled, you should also specify the name of the log files to safe_mysqld with --log, --log-update, or --log-slow-queries. Otherwise, both servers may be trying to write to the same log file.

WARNING: Normally you should never have two servers that update data in the same database! If your OS doesn't support fault-free system locking, this may lead to unpleasant surprises!

If you want to use another database directory for the second server, you can use the --datadir=path option to safe_mysqld.

NOTE also that starting several MySQL servers (mysqlds) in different machines and letting them access one data directory over NFS is generally a BAD IDEA! The problem is that the NFS will become the bottleneck with the speed. It is not meant for such use. And last but not least, you would still have to come up with a solution how to make sure that two or more mysqlds are not interfering with each other. At the moment there is no platform that would 100% reliable do the file locking (lockd daemon usually) in every situation. Yet there would be one more possible risk with NFS; it would make the work even more complicated for lockd daemon to handle. So make it easy for your self and forget about the idea. The working solution is to have one computer with an operating system that efficiently handles threads and have several CPUs in it.

When you want to connect to a MySQL server that is running with a different port than the port that is compiled into your client, you can use one of the following methods:

• Start the client with --host 'hostname' --port=port_number to connect with TCP/IP, or [--host localhost] --socket=file_name to connect via a Unix socket.
• In your C or Perl programs, you can give the port or socket arguments when connecting to the MySQL server.
• If your are using the Perl DBD::mysql module you can read the options from the MySQL option files. See section 4.1.2 my.cnf Option Files.

  $dsn = "DBI:mysql:test;mysql_read_default_group=client;mysql_read_default_file=/usr/local/mysql/data/my.cnf"
  $dbh = DBI->connect($dsn, $user, $password);
  

• Set the MYSQL_UNIX_PORT and MYSQL_TCP_PORT environment variables to point to the Unix socket and TCP/IP port before you start your clients. If you normally use a specific socket or port, you should place commands to set these environment variables in your `.login' file. See section H Environment Variables.
• Specify the default socket and TCP/IP port in the `.my.cnf' file in your home directory. See section 4.1.2 my.cnf Option Files.

4.2 General Security Issues and the MySQL Access Privilege System

MySQL has an advanced but non-standard security/privilege system. This section describes how it works.

4.2.1 General Security Guidelines

Anyone using MySQL on a computer connected to the Internet should read this section to avoid the most common security mistakes.

In discussing security, we emphasize the necessity of fully protecting the entire server host (not simply the MySQL server) against all types of applicable attacks: eavesdropping, altering, playback, and denial of service. We do not cover all aspects of availability and fault tolerance here.

MySQL uses security based on Access Control Lists (ACLs) for all connections, queries, and other operations that a user may attempt to perform. There is also some support for SSL-encrypted connections between MySQL clients and servers. Many of the concepts discussed here are not specific to MySQL at all; the same general ideas apply to almost all applications.

When running MySQL, follow these guidelines whenever possible:

• DON'T EVER GIVE ANYONE (EXCEPT THE MySQL ROOT USER) ACCESS TO THE user TABLE IN THE mysql DATABASE! The encrypted password is the real password in MySQL. If you know the password listed in the user table for a given user, you can easily log in as that user if you have access to the host listed for that account.
• Learn the MySQL access privilege system. The GRANT and REVOKE commands are used for controlling access to MySQL. Do not grant any more privileges than necessary. Never grant privileges to all hosts. Checklist:
  • Try mysql -u root. If you are able to connect successfully to the server without being asked for a password, you have problems. Anyone can connect to your MySQL server as the MySQL root user with full privileges! Review the MySQL installation instructions, paying particular attention to the item about setting a root password.
  • Use the command SHOW GRANTS and check to see who has access to what. Remove those privileges that are not necessary using the REVOKE command.
• Do not keep any plain-text passwords in your database. When your computer becomes compromised, the intruder can take the full list of passwords and use them. Instead use MD5() or another one-way hashing function.
• Do not choose passwords from dictionaries. There are special programs to break them. Even passwords like ``xfish98'' are very bad. Much better is ``duag98'' which contains the same word ``fish'' but typed one key to the left on a standard QWERTY keyboard. Another method is to use ``Mhall'' which is taken from the first characters of each word in the sentence ``Mary had a little lamb.'' This is easy to remember and type, but difficult to guess for someone who does not know it.
• Invest in a firewall. This protects you from at least 50% of all types of exploits in any software. Put MySQL behind the firewall or in a demilitarized zone (DMZ). Checklist:
  • Try to scan your ports from the Internet using a tool such as nmap. MySQL uses port 3306 by default. This port should be inaccessible from untrusted hosts. Another simple way to check whether or not your MySQL port is open is to try the following command from some remote machine, where server_host is the hostname of your MySQL server:

    shell> telnet server_host 3306
    

    If you get a connection and some garbage characters, the port is open, and should be closed on your firewall or router, unless you really have a good reason to keep it open. If telnet just hangs or the connection is refused, everything is OK; the port is blocked.
• Do not trust any data entered by your users. They can try to trick your code by entering special or escaped character sequences in Web forms, URLs, or whatever application you have built. Be sure that your application remains secure if a user enters something like ``; DROP DATABASE mysql;''. This is an extreme example, but large security leaks and data loss may occur as a result of hackers using similar techniques, if you do not prepare for them. Also remember to check numeric data. A common mistake is to protect only strings. Sometimes people think that if a database contains only publicly available data that it need not be protected. This is incorrect. At least denial-of-service type attacks can be performed on such databases. The simplest way to protect from this type of attack is to use apostrophes around the numeric constants: SELECT * FROM table WHERE ID='234' rather than SELECT * FROM table WHERE ID=234. MySQL automatically converts this string to a number and strips all non-numeric symbols from it. Checklist:
  • All Web applications:
    • Try to enter `'' and `"' in all your Web forms. If you get any kind of MySQL error, investigate the problem right away.
    • Try to modify any dynamic URLs by adding %22 (`"'), %23 (`#'), and %27 (`'') in the URL.
    • Try to modify datatypes in dynamic URLs from numeric ones to character ones containing characters from previous examples. Your application should be safe against this and similar attacks.
    • Try to enter characters, spaces, and special symbols instead of numbers in numeric fields. Your application should remove them before passing them to MySQL or your application should generate an error. Passing unchecked values to MySQL is very dangerous!
    • Check data sizes before passing them to MySQL.
    • Consider having your application connect to the database using a different user name than the one you use for administrative purposes. Do not give your applications any more access privileges than they need.
  • Users of PHP:
    • Check out the addslashes() function. As of PHP 4.0.3, a mysql_escape_string() function is available that is based on the function of the same name in the MySQL C API.
  • Users of MySQL C API:
    • Check out the mysql_escape_string() API call.
  • Users of MySQL++:
    • Check out the escape and quote modifiers for query streams.
  • Users of Perl DBI:
    • Check out the quote() method or use placeholders.
  • Users of Java JDBC:
    • Use a PreparedStatement object and placeholders.
• Do not transmit plain (unencrypted) data over the Internet. These data are accessible to everyone who has the time and ability to intercept it and use it for their own purposes. Instead, use an encrypted protocol such as SSL or SSH. MySQL supports internal SSL connections as of Version 3.23.9. SSH port-forwarding can be used to create an encrypted (and compressed) tunnel for the communication.
• Learn to use the tcpdump and strings utilities. For most cases, you can check whether or not MySQL data streams are unencrypted by issuing a command like the following:

  shell> tcpdump -l -i eth0 -w - src or dst port 3306 | strings
  

  (This works under Linux and should work with small modifications under other systems). Warning: If you do not see data this doesn't always actually mean that it is encrypted. If you need high security, you should consult with a security expert.

4.2.2 How to Make MySQL Secure Against Crackers

When you connect to a MySQL server, you normally should use a password. The password is not transmitted in clear text over the connection, however the encryption algorithm is not very strong, and with some effort a clever attacker can crack the password if he is able to sniff the traffic between the client and the server. If the connection between the client and the server goes through an untrusted network, you should use an SSH tunnel to encrypt the communication.

All other information is transferred as text that can be read by anyone who is able to watch the connection. If you are concerned about this, you can use the compressed protocol (in MySQL Version 3.22 and above) to make things much harder. To make things even more secure you should use ssh. You can find an open source ssh client at http://www.openssh.org, and a commercial ssh client at http://www.ssh.com. With this, you can get an encrypted TCP/IP connection between a MySQL server and a MySQL client.

To make a MySQL system secure, you should strongly consider the following suggestions:

• Use passwords for all MySQL users. Remember that anyone can log in as any other person as simply as mysql -u other_user db_name if other_user has no password. It is common behavior with client/server applications that the client may specify any user name. You can change the password of all users by editing the mysql_install_db script before you run it, or only the password for the MySQL root user like this:

  shell> mysql -u root mysql
  mysql> UPDATE user SET Password=PASSWORD('new_password')
             WHERE user='root';
  mysql> FLUSH PRIVILEGES;
  

• Don't run the MySQL daemon as the Unix root user. This is very dangerous, because any user with FILE privileges will be able to create files as root (for example, ~root/.bashrc). To prevent this, mysqld will refuse to run as root unless it is specified directly using a --user=root option. mysqld can be run as an ordinary unprivileged user instead. You can also create a new Unix user mysql to make everything even more secure. If you run mysqld as another Unix user, you don't need to change the root user name in the user table, because MySQL user names have nothing to do with Unix user names. To start mysqld as another Unix user, add a user line that specifies the user name to the [mysqld] group of the `/etc/my.cnf' option file or the `my.cnf' option file in the server's data directory. For example:

  [mysqld]
  user=mysql
  

  This will cause the server to start as the designated user whether you start it manually or by using safe_mysqld or mysql.server. For more details, see section A.3.2 How to Run MySQL As a Normal User.
• Don't support symlinks to tables (This can be disabled with the --skip-symlink option. This is especially important if you run mysqld as root as anyone that has write access to the mysqld data directories could then delete any file in the system! See section 5.6.1.2 Using Symbolic Links for Tables.
• Check that the Unix user that mysqld runs as is the only user with read/write privileges in the database directories.
• Don't give the process privilege to all users. The output of mysqladmin processlist shows the text of the currently executing queries, so any user who is allowed to execute that command might be able to see if another user issues an UPDATE user SET password=PASSWORD('not_secure') query. mysqld reserves an extra connection for users who have the process privilege, so that a MySQL root user can log in and check things even if all normal connections are in use.
• Don't give the file privilege to all users. Any user that has this privilege can write a file anywhere in the file system with the privileges of the mysqld daemon! To make this a bit safer, all files generated with SELECT ... INTO OUTFILE are readable to everyone, and you cannot overwrite existing files. The file privilege may also be used to read any file accessible to the Unix user that the server runs as. This could be abused, for example, by using LOAD DATA to load `/etc/passwd' into a table, which can then be read with SELECT.
• If you don't trust your DNS, you should use IP numbers instead of hostnames in the grant tables. In any case, you should be very careful about creating grant table entries using hostname values that contain wild cards!
• If you want to restrict the number of connections for a single user, you can do this by setting the max_user_connections variable in mysqld.

4.2.3 Startup Options for mysqld Concerning Security

The following mysqld options affect security:

--safe-show-database

With this option, SHOW DATABASES returns only those databases for which the user has some kind of privilege.

--safe-user-create

If this is enabled, an user can't create new users with the GRANT command, if the user doesn't have INSERT privilege to the mysql.user table. If you want to give a user access to just create new users with those privileges that the user has right to grant, you should give the user the following privilege:

GRANT INSERT(user) on mysql.user to 'user''hostname';

This will ensure that the user can't change any privilege columns directly, but has to use the GRANT command to give privileges to other users.

--skip-grant-tables

This option causes the server not to use the privilege system at all. This gives everyone full access to all databases! (You can tell a running server to start using the grant tables again by executing mysqladmin flush-privileges or mysqladmin reload.)

--skip-name-resolve

Hostnames are not resolved. All Host column values in the grant tables must be IP numbers or localhost.

--skip-networking

Don't allow TCP/IP connections over the network. All connections to mysqld must be made via Unix sockets. This option is unsuitable for systems that use MIT-pthreads, because the MIT-pthreads package doesn't support Unix sockets.

--skip-show-database

With this option, the SHOW DATABASES statement doesn't return anything.

4.2.4 What the Privilege System Does

The primary function of the MySQL privilege system is to authenticate a user connecting from a given host, and to associate that user with privileges on a database such as select, insert, update and delete.

Additional functionality includes the ability to have an anonymous user and to grant privileges for MySQL-specific functions such as LOAD DATA INFILE and administrative operations.

4.2.5 How the Privilege System Works

The MySQL privilege system ensures that all users may do exactly the things that they are supposed to be allowed to do. When you connect to a MySQL server, your identity is determined by the host from which you connect and the user name you specify. The system grants privileges according to your identity and what you want to do.

MySQL considers both your hostname and user name in identifying you because there is little reason to assume that a given user name belongs to the same person everywhere on the Internet. For example, the user bill who connects from whitehouse.gov need not be the same person as the user bill who connects from microsoft.com. MySQL handles this by allowing you to distinguish users on different hosts that happen to have the same name: you can grant bill one set of privileges for connections from whitehouse.gov, and a different set of privileges for connections from microsoft.com.

MySQL access control involves two stages:

• Stage 1: The server checks whether or not you are even allowed to connect.
• Stage 2: Assuming you can connect, the server checks each request you issue to see whether or not you have sufficient privileges to perform it. For example, if you try to select rows from a table in a database or drop a table from the database, the server makes sure you have the select privilege for the table or the drop privilege for the database.

The server uses the user, db, and host tables in the mysql database at both stages of access control. The fields in these grant tables are shown below:

For the second stage of access control (request verification), the server may, if the request involves tables, additionally consult the tables_priv and columns_priv tables. The fields in these tables are shown below:

Each grant table contains scope fields and privilege fields.

Scope fields determine the scope of each entry in the tables, that is, the context in which the entry applies. For example, a user table entry with Host and User values of 'thomas.loc.gov' and 'bob' would be used for authenticating connections made to the server by bob from the host thomas.loc.gov. Similarly, a db table entry with Host, User, and Db fields of 'thomas.loc.gov', 'bob' and 'reports' would be used when bob connects from the host thomas.loc.gov to access the reports database. The tables_priv and columns_priv tables contain scope fields indicating tables or table/column combinations to which each entry applies.

For access-checking purposes, comparisons of Host values are case insensitive. User, Password, Db, and Table_name values are case sensitive. Column_name values are case insensitive in MySQL Version 3.22.12 or later.

Privilege fields indicate the privileges granted by a table entry, that is, what operations can be performed. The server combines the information in the various grant tables to form a complete description of a user's privileges. The rules used to do this are described in section 4.2.9 Access Control, Stage 2: Request Verification.

Scope fields are strings, declared as shown below; the default value for each is the empty string:

In the user, db and host tables, all privilege fields are declared as ENUM('N','Y') -- each can have a value of 'N' or 'Y', and the default value is 'N'.

In the tables_priv and columns_priv tables, the privilege fields are declared as SET fields:

Briefly, the server uses the grant tables like this:

• The user table scope fields determine whether to allow or reject incoming connections. For allowed connections, any privileges granted in the user table indicate the user's global (superuser) privileges. These privileges apply to all databases on the server.
• The db and host tables are used together:
  • The db table scope fields determine which users can access which databases from which hosts. The privilege fields determine which operations are allowed.
  • The host table is used as an extension of the db table when you want a given db table entry to apply to several hosts. For example, if you want a user to be able to use a database from several hosts in your network, leave the Host value empty in the user's db table entry, then populate the host table with an entry for each of those hosts. This mechanism is described more detail in section 4.2.9 Access Control, Stage 2: Request Verification.
• The tables_priv and columns_priv tables are similar to the db table, but are more fine-grained: they apply at the table and column levels rather than at the database level.

Note that administrative privileges (reload, shutdown, etc.) are specified only in the user table. This is because administrative operations are operations on the server itself and are not database-specific, so there is no reason to list such privileges in the other grant tables. In fact, only the user table need be consulted to determine whether or not you can perform an administrative operation.

The file privilege is specified only in the user table, too. It is not an administrative privilege as such, but your ability to read or write files on the server host is independent of the database you are accessing.

The mysqld server reads the contents of the grant tables once, when it starts up. Changes to the grant tables take effect as indicated in section 4.3.3 When Privilege Changes Take Effect.

When you modify the contents of the grant tables, it is a good idea to make sure that your changes set up privileges the way you want. For help in diagnosing problems, see section 4.2.10 Causes of Access denied Errors. For advice on security issues, see section 4.2.2 How to Make MySQL Secure Against Crackers.

A useful diagnostic tool is the mysqlaccess script, which Yves Carlier has provided for the MySQL distribution. Invoke mysqlaccess with the --help option to find out how it works. Note that mysqlaccess checks access using only the user, db and host tables. It does not check table- or column-level privileges.

4.2.6 Privileges Provided by MySQL

Information about user privileges is stored in the user, db, host, tables_priv, and columns_priv tables in the mysql database (that is, in the database named mysql). The MySQL server reads the contents of these tables when it starts up and under the circumstances indicated in section 4.3.3 When Privilege Changes Take Effect.

The names used in this manual to refer to the privileges provided by MySQL are shown below, along with the table column name associated with each privilege in the grant tables and the context in which the privilege applies:

The select, insert, update, and delete privileges allow you to perform operations on rows in existing tables in a database.

SELECT statements require the select privilege only if they actually retrieve rows from a table. You can execute certain SELECT statements even without permission to access any of the databases on the server. For example, you could use the mysql client as a simple calculator:

mysql> SELECT 1+1;
mysql> SELECT PI()*2;

The index privilege allows you to create or drop (remove) indexes.

The alter privilege allows you to use ALTER TABLE.

The create and drop privileges allow you to create new databases and tables, or to drop (remove) existing databases and tables.

Note that if you grant the drop privilege for the mysql database to a user, that user can drop the database in which the MySQL access privileges are stored!

The grant privilege allows you to give to other users those privileges you yourself possess.

The file privilege gives you permission to read and write files on the server using the LOAD DATA INFILE and SELECT ... INTO OUTFILE statements. Any user to whom this privilege is granted can read or write any file that the MySQL server can read or write.

The remaining privileges are used for administrative operations, which are performed using the mysqladmin program. The table below shows which mysqladmin commands each administrative privilege allows you to execute:

The reload command tells the server to re-read the grant tables. The refresh command flushes all tables and opens and closes the log files. flush-privileges is a synonym for reload. The other flush-* commands perform functions similar to refresh but are more limited in scope, and may be preferable in some instances. For example, if you want to flush just the log files, flush-logs is a better choice than refresh.

The shutdown command shuts down the server.

The processlist command displays information about the threads executing within the server. The kill command kills server threads. You can always display or kill your own threads, but you need the process privilege to display or kill threads initiated by other users. See section 4.5.4 KILL Syntax.

It is a good idea in general to grant privileges only to those users who need them, but you should exercise particular caution in granting certain privileges:

• The grant privilege allows users to give away their privileges to other users. Two users with different privileges and with the grant privilege are able to combine privileges.
• The alter privilege may be used to subvert the privilege system by renaming tables.
• The file privilege can be abused to read any world-readable file on the server into a database table, the contents of which can then be accessed using SELECT. This includes the contents of all databases hosted by the server!
• The shutdown privilege can be abused to deny service to other users entirely, by terminating the server.
• The process privilege can be used to view the plain text of currently executing queries, including queries that set or change passwords.
• Privileges on the mysql database can be used to change passwords and other access privilege information. (Passwords are stored encrypted, so a malicious user cannot simply read them to know the plain text password). If they can access the mysql.user password column, they can use it to log into the MySQL server for the given user. (With sufficient privileges, the same user can replace a password with a different one.)

There are some things that you cannot do with the MySQL privilege system:

• You cannot explicitly specify that a given user should be denied access. That is, you cannot explicitly match a user and then refuse the connection.
• You cannot specify that a user has privileges to create or drop tables in a database but not to create or drop the database itself.

4.2.7 Connecting to the MySQL Server

MySQL client programs generally require that you specify connection parameters when you want to access a MySQL server: the host you want to connect to, your user name, and your password. For example, the mysql client can be started like this (optional arguments are enclosed between `[' and `]'):

shell> mysql [-h host_name] [-u user_name] [-pyour_pass]

Alternate forms of the -h, -u, and -p options are --host=host_name, --user=user_name, and --password=your_pass. Note that there is no space between -p or --password= and the password following it.

NOTE: Specifying a password on the command line is not secure! Any user on your system may then find out your password by typing a command like: ps auxww. See section 4.1.2 my.cnf Option Files.

mysql uses default values for connection parameters that are missing from the command line:

• The default hostname is localhost.
• The default user name is your Unix login name.
• No password is supplied if -p is missing.

Thus, for a Unix user joe, the following commands are equivalent:

shell> mysql -h localhost -u joe
shell> mysql -h localhost
shell> mysql -u joe
shell> mysql

Other MySQL clients behave similarly.

On Unix systems, you can specify different default values to be used when you make a connection, so that you need not enter them on the command line each time you invoke a client program. This can be done in a couple of ways:

• You can specify connection parameters in the [client] section of the `.my.cnf' configuration file in your home directory. The relevant section of the file might look like this:

  [client]
  host=host_name
  user=user_name
  password=your_pass
  

  See section 4.1.2 my.cnf Option Files.
• You can specify connection parameters using environment variables. The host can be specified for mysql using MYSQL_HOST. The MySQL user name can be specified using USER (this is for Windows only). The password can be specified using MYSQL_PWD (but this is insecure; see the next section). See section H Environment Variables.

4.2.8 Access Control, Stage 1: Connection Verification

When you attempt to connect to a MySQL server, the server accepts or rejects the connection based on your identity and whether or not you can verify your identity by supplying the correct password. If not, the server denies access to you completely. Otherwise, the server accepts the connection, then enters Stage 2 and waits for requests.

Your identity is based on two pieces of information:

• The host from which you connect
• Your MySQL user name

Identity checking is performed using the three user table scope fields (Host, User, and Password). The server accepts the connection only if a user table entry matches your hostname and user name, and you supply the correct password.

Values in the user table scope fields may be specified as follows:

• A Host value may be a hostname or an IP number, or 'localhost' to indicate the local host.
• You can use the wild-card characters `%' and `_' in the Host field.
• A Host value of '%' matches any hostname.
• A blank Host value means that the privilege should be anded with the entry in the host table that matches the given host name. You can find more information about this in the next chapter.
• As of MySQL Version 3.23, for Host values specified as IP numbers, you can specify a netmask indicating how many address bits to use for the network number. For example:

  GRANT ALL PRIVILEGES on db.* to david@'192.58.197.0/255.255.255.0';
  

  This will allow everyone to connect from an IP where the following is true:

  user_ip & netmask = host_ip.
  

  In the above example all IP:s in the interval 192.58.197.0 - 192.58.197.255 can connect to the MySQL server.
• Wild-card characters are not allowed in the User field, but you can specify a blank value, which matches any name. If the user table entry that matches an incoming connection has a blank user name, the user is considered to be the anonymous user (the user with no name), rather than the name that the client actually specified. This means that a blank user name is used for all further access checking for the duration of the connection (that is, during Stage 2).
• The Password field can be blank. This does not mean that any password matches, it means the user must connect without specifying a password.

Non-blank Password values represent encrypted passwords. MySQL does not store passwords in plaintext form for anyone to see. Rather, the password supplied by a user who is attempting to connect is encrypted (using the PASSWORD() function). The encrypted password is then used when the client/server is checking if the password is correct (This is done without the encrypted password ever traveling over the connection.) Note that from MySQL's point of view the encrypted password is the REAL password, so you should not give anyone access to it! In particular, don't give normal users read access to the tables in the mysql database!

The examples below show how various combinations of Host and User values in user table entries apply to incoming connections:

Because you can use IP wild-card values in the Host field (for example, '144.155.166.%' to match every host on a subnet), there is the possibility that someone might try to exploit this capability by naming a host 144.155.166.somewhere.com. To foil such attempts, MySQL disallows matching on hostnames that start with digits and a dot. Thus, if you have a host named something like 1.2.foo.com, its name will never match the Host column of the grant tables. Only an IP number can match an IP wild-card value.

An incoming connection may be matched by more than one entry in the user table. For example, a connection from thomas.loc.gov by fred would be matched by several of the entries just shown above. How does the server choose which entry to use if more than one matches? The server resolves this question by sorting the user table after reading it at startup time, then looking through the entries in sorted order when a user attempts to connect. The first matching entry is the one that is used.

user table sorting works as follows. Suppose the user table looks like this:

+-----------+----------+-
| Host      | User     | ...
+-----------+----------+-
| %         | root     | ...
| %         | jeffrey  | ...
| localhost | root     | ...
| localhost |          | ...
+-----------+----------+-

When the server reads in the table, it orders the entries with the most-specific Host values first ('%' in the Host column means ``any host'' and is least specific). Entries with the same Host value are ordered with the most-specific User values first (a blank User value means ``any user'' and is least specific). The resulting sorted user table looks like this:

+-----------+----------+-
| Host      | User     | ...
+-----------+----------+-
| localhost | root     | ...
| localhost |          | ...
| %         | jeffrey  | ...
| %         | root     | ...
+-----------+----------+-

When a connection is attempted, the server looks through the sorted entries and uses the first match found. For a connection from localhost by jeffrey, the entries with 'localhost' in the Host column match first. Of those, the entry with the blank user name matches both the connecting hostname and user name. (The '%'/'jeffrey' entry would have matched, too, but it is not the first match in the table.)

Here is another example. Suppose the user table looks like this:

+----------------+----------+-
| Host           | User     | ...
+----------------+----------+-
| %              | jeffrey  | ...
| thomas.loc.gov |          | ...
+----------------+----------+-

The sorted table looks like this:

+----------------+----------+-
| Host           | User     | ...
+----------------+----------+-
| thomas.loc.gov |          | ...
| %              | jeffrey  | ...
+----------------+----------+-

A connection from thomas.loc.gov by jeffrey is matched by the first entry, whereas a connection from whitehouse.gov by jeffrey is matched by the second.

A common misconception is to think that for a given user name, all entries that explicitly name that user will be used first when the server attempts to find a match for the connection. This is simply not true. The previous example illustrates this, where a connection from thomas.loc.gov by jeffrey is first matched not by the entry containing 'jeffrey' as the User field value, but by the entry with no user name!

If you have problems connecting to the server, print out the user table and sort it by hand to see where the first match is being made.

4.2.9 Access Control, Stage 2: Request Verification

Once you establish a connection, the server enters Stage 2. For each request that comes in on the connection, the server checks whether you have sufficient privileges to perform it, based on the type of operation you wish to perform. This is where the privilege fields in the grant tables come into play. These privileges can come from any of the user, db, host, tables_priv, or columns_priv tables. The grant tables are manipulated with GRANT and REVOKE commands. See section 4.3.1 GRANT and REVOKE Syntax. (You may find it helpful to refer to section 4.2.5 How the Privilege System Works, which lists the fields present in each of the grant tables.)

The user table grants privileges that are assigned to you on a global basis and that apply no matter what the current database is. For example, if the user table grants you the delete privilege, you can delete rows from any database on the server host! In other words, user table privileges are superuser privileges. It is wise to grant privileges in the user table only to superusers such as server or database administrators. For other users, you should leave the privileges in the user table set to 'N' and grant privileges on a database-specific basis only, using the db and host tables.

The db and host tables grant database-specific privileges. Values in the scope fields may be specified as follows:

• The wild-card characters `%' and `_' can be used in the Host and Db fields of either table.
• A '%' Host value in the db table means ``any host.'' A blank Host value in the db table means ``consult the host table for further information.''
• A '%' or blank Host value in the host table means ``any host.''
• A '%' or blank Db value in either table means ``any database.''
• A blank User value in either table matches the anonymous user.

The db and host tables are read in and sorted when the server starts up (at the same time that it reads the user table). The db table is sorted on the Host, Db, and User scope fields, and the host table is sorted on the Host and Db scope fields. As with the user table, sorting puts the most-specific values first and least-specific values last, and when the server looks for matching entries, it uses the first match that it finds.

The tables_priv and columns_priv tables grant table- and column-specific privileges. Values in the scope fields may be specified as follows:

• The wild-card characters `%' and `_' can be used in the Host field of either table.
• A '%' or blank Host value in either table means ``any host.''
• The Db, Table_name and Column_name fields cannot contain wild cards or be blank in either table.

The tables_priv and columns_priv tables are sorted on the Host, Db, and User fields. This is similar to db table sorting, although the sorting is simpler because only the Host field may contain wild cards.

The request verification process is described below. (If you are familiar with the access-checking source code, you will notice that the description here differs slightly from the algorithm used in the code. The description is equivalent to what the code actually does; it differs only to make the explanation simpler.)

For administrative requests (shutdown, reload, etc.), the server checks only the user table entry, because that is the only table that specifies administrative privileges. Access is granted if the entry allows the requested operation and denied otherwise. For example, if you want to execute mysqladmin shutdown but your user table entry doesn't grant the shutdown privilege to you, access is denied without even checking the db or host tables. (They contain no Shutdown_priv column, so there is no need to do so.)

For database-related requests (insert, update, etc.), the server first checks the user's global (superuser) privileges by looking in the user table entry. If the entry allows the requested operation, access is granted. If the global privileges in the user table are insufficient, the server determines the user's database-specific privileges by checking the db and host tables:

1. The server looks in the db table for a match on the Host, Db, and User fields. The Host and User fields are matched to the connecting user's hostname and MySQL user name. The Db field is matched to the database the user wants to access. If there is no entry for the Host and User, access is denied.
2. If there is a matching db table entry and its Host field is not blank, that entry defines the user's database-specific privileges.
3. If the matching db table entry's Host field is blank, it signifies that the host table enumerates which hosts should be allowed access to the database. In this case, a further lookup is done in the host table to find a match on the Host and Db fields. If no host table entry matches, access is denied. If there is a match, the user's database-specific privileges are computed as the intersection (not the union!) of the privileges in the db and host table entries, that is, the privileges that are 'Y' in both entries. (This way you can grant general privileges in the db table entry and then selectively restrict them on a host-by-host basis using the host table entries.)

After determining the database-specific privileges granted by the db and host table entries, the server adds them to the global privileges granted by the user table. If the result allows the requested operation, access is granted. Otherwise, the server checks the user's table and column privileges in the tables_priv and columns_priv tables and adds those to the user's privileges. Access is allowed or denied based on the result.

Expressed in boolean terms, the preceding description of how a user's privileges are calculated may be summarized like this:

global privileges
OR (database privileges AND host privileges)
OR table privileges
OR column privileges

It may not be apparent why, if the global user entry privileges are initially found to be insufficient for the requested operation, the server adds those privileges to the database-, table-, and column-specific privileges later. The reason is that a request might require more than one type of privilege. For example, if you execute an INSERT ... SELECT statement, you need both insert and select privileges. Your privileges might be such that the user table entry grants one privilege and the db table entry grants the other. In this case, you have the necessary privileges to perform the request, but the server cannot tell that from either table by itself; the privileges granted by the entries in both tables must be combined.

The host table can be used to maintain a list of secure servers.

At TcX, the host table contains a list of all machines on the local network. These are granted all privileges.

You can also use the host table to indicate hosts that are not secure. Suppose you have a machine public.your.domain that is located in a public area that you do not consider secure. You can allow access to all hosts on your network except that machine by using host table entries like this:

+--------------------+----+-
| Host               | Db | ...
+--------------------+----+-
| public.your.domain | %  | ... (all privileges set to 'N')
| %.your.domain      | %  | ... (all privileges set to 'Y')
+--------------------+----+-

Naturally, you should always test your entries in the grant tables (for example, using mysqlaccess) to make sure your access privileges are actually set up the way you think they are.

4.2.10 Causes of Access denied Errors

If you encounter Access denied errors when you try to connect to the MySQL server, the list below indicates some courses of action you can take to correct the problem:

• After installing MySQL, did you run the mysql_install_db script to set up the initial grant table contents? If not, do so. See section 4.3.4 Setting Up the Initial MySQL Privileges. Test the initial privileges by executing this command:

  shell> mysql -u root test
  

  The server should let you connect without error. You should also make sure you have a file `user.MYD' in the MySQL database directory. Ordinarily, this is `PATH/var/mysql/user.MYD', where PATH is the pathname to the MySQL installation root.
• After a fresh installation, you should connect to the server and set up your users and their access permissions:

  shell> mysql -u root mysql
  

  The server should let you connect because the MySQL root user has no password initially. That is also a security risk, so setting the root password is something you should do while you're setting up your other MySQL users. If you try to connect as root and get this error:

  Access denied for user: '@unknown' to database mysql
  

  this means that you don't have an entry in the user table with a User column value of 'root' and that mysqld cannot resolve the hostname for your client. In this case, you must restart the server with the --skip-grant-tables option and edit your `/etc/hosts' or `\windows\hosts' file to add an entry for your host.
• If you get an error like the following:

  shell> mysqladmin -u root -pxxxx ver
  Access denied for user: 'root@localhost' (Using password: YES)
  

  It means that you are using a wrong password. See section 4.3.6 Setting Up Passwords. If you have forgot the root password, you can restart mysqld with --skip-grant-tables to change the password. You can find more about this option later on in this manual section. If you get the above error even if you haven't specified a password, this means that you a wrong password in some my.ini file. See section 4.1.2 my.cnf Option Files. You can avoid using option files with the --no-defaults option, as follows:

  shell> mysqladmin --no-defaults -u root ver
  

• If you updated an existing MySQL installation from a version earlier than Version 3.22.11 to Version 3.22.11 or later, did you run the mysql_fix_privilege_tables script? If not, do so. The structure of the grant tables changed with MySQL Version 3.22.11 when the GRANT statement became functional.
• If your privileges seem to have changed in the middle of a session, it may be that a superuser has changed them. Reloading the grant tables affects new client connections, but it also affects existing connections as indicated in section 4.3.3 When Privilege Changes Take Effect.
• If you can't get your password to work, remember that you must use the PASSWORD() function if you set the password with the INSERT, UPDATE, or SET PASSWORD statements. The PASSWORD() function is unnecessary if you specify the password using the GRANT ... INDENTIFIED BY statement or the mysqladmin password command. See section 4.3.6 Setting Up Passwords.
• localhost is a synonym for your local hostname, and is also the default host to which clients try to connect if you specify no host explicitly. However, connections to localhost do not work if you are running on a system that uses MIT-pthreads (localhost connections are made using Unix sockets, which are not supported by MIT-pthreads). To avoid this problem on such systems, you should use the --host option to name the server host explicitly. This will make a TCP/IP connection to the mysqld server. In this case, you must have your real hostname in user table entries on the server host. (This is true even if you are running a client program on the same host as the server.)
• If you get an Access denied error when trying to connect to the database with mysql -u user_name db_name, you may have a problem with the user table. Check this by executing mysql -u root mysql and issuing this SQL statement:

  mysql> SELECT * FROM user;
  

  The result should include an entry with the Host and User columns matching your computer's hostname and your MySQL user name.
• The Access denied error message will tell you who you are trying to log in as, the host from which you are trying to connect, and whether or not you were using a password. Normally, you should have one entry in the user table that exactly matches the hostname and user name that were given in the error message. For example if you get an error message that contains Using password: NO, this means that you tried to login without an password.
• If you get the following error when you try to connect from a different host than the one on which the MySQL server is running, then there is no row in the user table that matches that host:

  Host ... is not allowed to connect to this MySQL server
  

  You can fix this by using the command-line tool mysql (on the server host!) to add a row to the user, db, or host table for the user/hostname combination from which you are trying to connect and then execute mysqladmin flush-privileges. If you are not running MySQL Version 3.22 and you don't know the IP number or hostname of the machine from which you are connecting, you should put an entry with '%' as the Host column value in the user table and restart mysqld with the --log option on the server machine. After trying to connect from the client machine, the information in the MySQL log will indicate how you really did connect. (Then replace the '%' in the user table entry with the actual hostname that shows up in the log. Otherwise, you'll have a system that is insecure.) Another reason for this error on Linux is that you are using a binary MySQL version that is compiled with a different glibc version than the one you are using. In this case you should either upgrade your OS/glibc or download the source MySQL version and compile this yourself. A source RPM is normally trivial to compile and install, so this isn't a big problem.
• If you get an error message where the hostname is not shown or where the hostname is an IP, even if you try to connect with a hostname:

  shell> mysqladmin -u root -pxxxx -h some-hostname ver
  Access denied for user: 'root' (Using password: YES)
  

  This means that MySQL got some error when trying to resolve the IP to a hostname. In this case you can execute mysqladmin flush-hosts to reset the internal DNS cache. See section 5.5.5 How MySQL uses DNS. Some permanent solutions are:
  • Try to find out what is wrong with your DNS server and fix this.
  • Specify IPs instead of hostnames in the MySQL privilege tables.
  • Start mysqld with --skip-name-resolve.
  • Start mysqld with --skip-host-cache.
  • Connect to localhost if you are running the server and the client on the same machine.
  • Put the client machine names in /etc/hosts.
• If mysql -u root test works but mysql -h your_hostname -u root test results in Access denied, then you may not have the correct name for your host in the user table. A common problem here is that the Host value in the user table entry specifies an unqualified hostname, but your system's name resolution routines return a fully qualified domain name (or vice-versa). For example, if you have an entry with host 'tcx' in the user table, but your DNS tells MySQL that your hostname is 'tcx.subnet.se', the entry will not work. Try adding an entry to the user table that contains the IP number of your host as the Host column value. (Alternatively, you could add an entry to the user table with a Host value that contains a wild card--for example, 'tcx.%'. However, use of hostnames ending with `%' is insecure and is not recommended!)
• If mysql -u user_name test works but mysql -u user_name other_db_name doesn't work, you don't have an entry for other_db_name listed in the db table.
• If mysql -u user_name db_name works when executed on the server machine, but mysql -u host_name -u user_name db_name doesn't work when executed on another client machine, you don't have the client machine listed in the user table or the db table.
• If you can't figure out why you get Access denied, remove from the user table all entries that have Host values containing wild cards (entries that contain `%' or `_'). A very common error is to insert a new entry with Host='%' and User='some user', thinking that this will allow you to specify localhost to connect from the same machine. The reason that this doesn't work is that the default privileges include an entry with Host='localhost' and User=''. Because that entry has a Host value 'localhost' that is more specific than '%', it is used in preference to the new entry when connecting from localhost! The correct procedure is to insert a second entry with Host='localhost' and User='some_user', or to remove the entry with Host='localhost' and User=''.
• If you get the following error, you may have a problem with the db or host table:

  Access to database denied
  

  If the entry selected from the db table has an empty value in the Host column, make sure there are one or more corresponding entries in the host table specifying which hosts the db table entry applies to. If you get the error when using the SQL commands SELECT ... INTO OUTFILE or LOAD DATA INFILE, your entry in the user table probably doesn't have the file privilege enabled.
• Remember that client programs will use connection parameters specified in configuration files or environment variables. See section H Environment Variables. If a client seems to be sending the wrong default connection parameters when you don't specify them on the command line, check your environment and the `.my.cnf' file in your home directory. You might also check the system-wide MySQL configuration files, though it is far less likely that client connection parameters will be specified there. See section 4.1.2 my.cnf Option Files. If you get Access denied when you run a client without any options, make sure you haven't specified an old password in any of your option files! See section 4.1.2 my.cnf Option Files.
• If you make changes to the grant tables directly (using an INSERT or UPDATE statement) and your changes seem to be ignored, remember that you must issue a FLUSH PRIVILEGES statement or execute a mysqladmin flush-privileges command to cause the server to re-read the privilege tables. Otherwise your changes have no effect until the next time the server is restarted. Remember that after you set the root password with an UPDATE command, you won't need to specify it until after you flush the privileges, because the server won't know you've changed the password yet!
• If you have access problems with a Perl, PHP, Python, or ODBC program, try to connect to the server with mysql -u user_name db_name or mysql -u user_name -pyour_pass db_name. If you are able to connect using the mysql client, there is a problem with your program and not with the access privileges. (Note that there is no space between -p and the password; you can also use the --password=your_pass syntax to specify the password. If you use the -p option alone, MySQL will prompt you for the password.)
• For testing, start the mysqld daemon with the --skip-grant-tables option. Then you can change the MySQL grant tables and use the mysqlaccess script to check whether or not your modifications have the desired effect. When you are satisfied with your changes, execute mysqladmin flush-privileges to tell the mysqld server to start using the new grant tables. Note: Reloading the grant tables overrides the --skip-grant-tables option. This allows you to tell the server to begin using the grant tables again without bringing it down and restarting it.
• If everything else fails, start the mysqld daemon with a debugging option (for example, --debug=d,general,query). This will print host and user information about attempted connections, as well as information about each command issued. See section G.1.2 Creating trace files.
• If you have any other problems with the MySQL grant tables and feel you must post the problem to the mailing list, always provide a dump of the MySQL grant tables. You can dump the tables with the mysqldump mysql command. As always, post your problem using the mysqlbug script. See section 1.2.22.3 How to Report Bugs or Problems. In some cases you may need to restart mysqld with --skip-grant-tables to run mysqldump.

4.3 MySQL User Account Management

4.3.1 GRANT and REVOKE Syntax

GRANT priv_type [(column_list)] [, priv_type [(column_list)] ...]
    ON {tbl_name | * | *.* | db_name.*}
    TO user_name [IDENTIFIED BY 'password']
        [, user_name [IDENTIFIED BY 'password'] ...]
    [REQUIRE {SSL|X509} [ISSUER issuer] [SUBJECT subject]]
    [WITH GRANT OPTION]

REVOKE priv_type [(column_list)] [, priv_type [(column_list)] ...]
    ON {tbl_name | * | *.* | db_name.*}
    FROM user_name [, user_name ...]

GRANT is implemented in MySQL Version 3.22.11 or later. For earlier MySQL versions, the GRANT statement does nothing.

The GRANT and REVOKE commands allow system administrators to create users and grant and revoke rights to MySQL users at four privilege levels:

Global level

Global privileges apply to all databases on a given server. These privileges are stored in the mysql.user table.

Database level

Database privileges apply to all tables in a given database. These privileges are stored in the mysql.db and mysql.host tables.

Table level

Table privileges apply to all columns in a given table. These privileges are stored in the mysql.tables_priv table.

Column level

Column privileges apply to single columns in a given table. These privileges are stored in the mysql.columns_priv table.

If you give a grant for a users that doesn't exists, that user is created. For examples of how GRANT works, see section 4.3.5 Adding New Users to MySQL.

For the GRANT and REVOKE statements, priv_type may be specified as any of the following:

ALL PRIVILEGES      FILE                RELOAD
ALTER               INDEX               SELECT
CREATE              INSERT              SHUTDOWN
DELETE              PROCESS             UPDATE
DROP                REFERENCES          USAGE

ALL is a synonym for ALL PRIVILEGES. REFERENCES is not yet implemented. USAGE is currently a synonym for ``no privileges.'' It can be used when you want to create a user that has no privileges.

To revoke the grant privilege from a user, use a priv_type value of GRANT OPTION:

REVOKE GRANT OPTION ON ... FROM ...;

The only priv_type values you can specify for a table are SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, GRANT, INDEX, and ALTER.

The only priv_type values you can specify for a column (that is, when you use a column_list clause) are SELECT, INSERT, and UPDATE.

You can set global privileges by using ON *.* syntax. You can set database privileges by using ON db_name.* syntax. If you specify ON * and you have a current database, you will set the privileges for that database. (WARNING: If you specify ON * and you don't have a current database, you will affect the global privileges!)

In order to accommodate granting rights to users from arbitrary hosts, MySQL supports specifying the user_name value in the form user@host. If you want to specify a user string containing special characters (such as `-'), or a host string containing special characters or wild-card characters (such as `%'), you can quote the user or host name (for example, 'test-user'@'test-hostname').

You can specify wild cards in the hostname. For example, user@"%.loc.gov" applies to user for any host in the loc.gov domain, and user@"144.155.166.%" applies to user for any host in the 144.155.166 class C subnet.

The simple form user is a synonym for user@"%". NOTE: If you allow anonymous users to connect to the MySQL server (which is the default), you should also add all local users as user@localhost because otherwise the anonymous user entry for the local host in the mysql.user table will be used when the user tries to log into the MySQL server from the local machine! Anonymous users are defined by inserting entries with User='' into the mysql.user table. You can verify if this applies to you by executing this query:

mysql> SELECT Host,User FROM mysql.user WHERE User='';

For the moment, GRANT only supports host, table, database, and column names up to 60 characters long. A user name can be up to 16 characters.

The privileges for a table or column are formed from the logical OR of the privileges at each of the four privilege levels. For example, if the mysql.user table specifies that a user has a global select privilege, this can't be denied by an entry at the database, table, or column level.

The privileges for a column can be calculated as follows:

global privileges
OR (database privileges AND host privileges)
OR table privileges
OR column privileges

In most cases, you grant rights to a user at only one of the privilege levels, so life isn't normally as complicated as above. The details of the privilege-checking procedure are presented in section 4.2 General Security Issues and the MySQL Access Privilege System.

If you grant privileges for a user/hostname combination that does not exist in the mysql.user table, an entry is added and remains there until deleted with a DELETE command. In other words, GRANT may create user table entries, but REVOKE will not remove them; you must do that explicitly using DELETE.

In MySQL Version 3.22.12 or later, if a new user is created or if you have global grant privileges, the user's password will be set to the password specified by the IDENTIFIED BY clause, if one is given. If the user already had a password, it is replaced by the new one.

WARNING: If you create a new user but do not specify an IDENTIFIED BY clause, the user has no password. This is insecure.

Passwords can also be set with the SET PASSWORD command. See section 5.5.6 SET Syntax.

If you grant privileges for a database, an entry in the mysql.db table is created if needed. When all privileges for the database have been removed with REVOKE, this entry is deleted.

If a user doesn't have any privileges on a table, the table is not displayed when the user requests a list of tables (for example, with a SHOW TABLES statement).

The WITH GRANT OPTION clause gives the user the ability to give to other users any privileges the user has at the specified privilege level. You should be careful to whom you give the grant privilege, as two users with different privileges may be able to join privileges!

You cannot grant another user a privilege you don't have yourself; the grant privilege allows you to give away only those privileges you possess.

Be aware that when you grant a user the grant privilege at a particular privilege level, any privileges the user already possesses (or is given in the future!) at that level are also grantable by that user. Suppose you grant a user the insert privilege on a database. If you then grant the select privilege on the database and specify WITH GRANT OPTION, the user can give away not only the select privilege, but also insert. If you then grant the update privilege to the user on the database, the user can give away the insert, select and update.

You should not grant alter privileges to a normal user. If you do that, the user can try to subvert the privilege system by renaming tables!

Note that if you are using table or column privileges for even one user, the server examines table and column privileges for all users and this will slow down MySQL a bit.

When mysqld starts, all privileges are read into memory. Database, table, and column privileges take effect at once, and user-level privileges take effect the next time the user connects. Modifications to the grant tables that you perform using GRANT or REVOKE are noticed by the server immediately. If you modify the grant tables manually (using INSERT, UPDATE, etc.), you should execute a FLUSH PRIVILEGES statement or run mysqladmin flush-privileges to tell the server to reload the grant tables. See section 4.3.3 When Privilege Changes Take Effect.

The biggest differences between the ANSI SQL and MySQL versions of GRANT are:

• In MySQL privileges are given for an username + hostname combination and not only for an username.
• ANSI SQL doesn't have global or database-level privileges, and ANSI SQL doesn't support all privilege types that MySQL supports. MySQL doesn't support the ANSI SQL TRIGGER, EXECUTE or UNDER privileges.
• ANSI SQL privileges are structured in a hierarchal manner. If you remove an user, all privileges the user has granted are revoked. In MySQL the granted privileges are not automatically revoked, but you have to revoke these yourself if needed.
• If you in MySQL have the INSERT grant on only part of the columns in a table, you can execute INSERT statements on the table; The columns for which you don't have the INSERT privilege will set to their default values. ANSI SQL requires you to have the INSERT privilege on all columns.
• When you drop a table in ANSI SQL, all privileges for the table are revoked. If you revoke a privilege in ANSI SQL, all privileges that were granted based on this privilege are also revoked. In MySQL, privileges can be dropped only with explicit REVOKE commands or by manipulating the MySQL grant tables.

4.3.2 MySQL User Names and Passwords

There are several distinctions between the way user names and passwords are used by MySQL and the way they are used by Unix or Windows:

• User names, as used by MySQL for authentication purposes, have nothing to do with Unix user names (login names) or Windows user names. Most MySQL clients by default try to log in using the current Unix user name as the MySQL user name, but that is for convenience only. Client programs allow a different name to be specified with the -u or --user options. This means that you can't make a database secure in any way unless all MySQL user names have passwords. Anyone may attempt to connect to the server using any name, and they will succeed if they specify any name that doesn't have a password.
• MySQL user names can be up to 16 characters long; Unix user names typically are limited to 8 characters.
• MySQL passwords have nothing to do with Unix passwords. There is no necessary connection between the password you use to log in to a Unix machine and the password you use to access a database on that machine.
• MySQL encrypts passwords using a different algorithm than the one used during the Unix login process. See the descriptions of the PASSWORD() and ENCRYPT() functions in section 6.3.5.2 Miscellaneous Functions. Note that even if the password is stored 'scrambled', and knowing your 'scrambled' password is enough to be able to connect to the MySQL server!

MySQL users and they privileges are normally created with the GRANT command. See section 4.3.1 GRANT and REVOKE Syntax.

When you login to a MySQL server with a command line client you should specify the password with --password=your-password. See section 4.2.7 Connecting to the MySQL Server.

mysql --user=monty --password=guess database_name

If you want the client to prompt for a password, you should use --password without any argument

mysql --user=monty --password database_name

or the short form:

mysql -u monty -p database_name

Note that in the last example the password is NOT 'database_name'.

If you want to use the -p option to supply a password you should do like this:

mysql -u monty -pguess database_name

On some system the library call that MySQL uses to prompt for a password will automatically cut the password to 8 characters. Internally MySQL doesn't have any limit for the length of the password.

4.3.3 When Privilege Changes Take Effect

When mysqld starts, all grant table contents are read into memory and become effective at that point.

Modifications to the grant tables that you perform using GRANT, REVOKE, or SET PASSWORD are noticed by the server immediately.

If you modify the grant tables manually (using INSERT, UPDATE, etc.), you should execute a FLUSH PRIVILEGES statement or run mysqladmin flush-privileges or mysqladmin reload to tell the server to reload the grant tables. Otherwise your changes will have no effect until you restart the server. If you change the grant tables manually but forget to reload the privileges, you will be wondering why your changes don't seem to make any difference!

When the server notices that the grant tables have been changed, existing client connections are affected as follows:

• Table and column privilege changes take effect with the client's next request.
• Database privilege changes take effect at the next USE db_name command.

Global privilege changes and password changes take effect the next time the client connects.

4.3.4 Setting Up the Initial MySQL Privileges

After installing MySQL, you set up the initial access privileges by running scripts/mysql_install_db. See section 2.3.1 Quick Installation Overview. The mysql_install_db script starts up the mysqld server, then initializes the grant tables to contain the following set of privileges:

• The MySQL root user is created as a superuser who can do anything. Connections must be made from the local host. NOTE: The initial root password is empty, so anyone can connect as root without a password and be granted all privileges.
• An anonymous user is created that can do anything with databases that have a name of 'test' or starting with 'test_'. Connections must be made from the local host. This means any local user can connect without a password and be treated as the anonymous user.
• Other privileges are denied. For example, normal users can't use mysqladmin shutdown or mysqladmin processlist.

NOTE: The default privileges are different for Windows. See section 2.6.2.3 Running MySQL on Windows.

Because your installation is initially wide open, one of the first things you should do is specify a password for the MySQL root user. You can do this as follows (note that you specify the password using the PASSWORD() function):

shell> mysql -u root mysql
mysql> UPDATE user SET Password=PASSWORD('new_password')
           WHERE user='root';
mysql> FLUSH PRIVILEGES;

You can, in MySQL Version 3.22 and above, use the SET PASSWORD statement:

shell> mysql -u root mysql
mysql> SET PASSWORD FOR root=PASSWORD('new_password');

Another way to set the password is by using the mysqladmin command:

shell> mysqladmin -u root password new_password

Only users with write/update access to the mysql database can change the password for others users. All normal users (not anonymous ones) can only change their own password with either of the above commands or with SET PASSWORD=PASSWORD('new password').

Note that if you update the password in the user table directly using the first method, you must tell the server to re-read the grant tables (with FLUSH PRIVILEGES), because the change will go unnoticed otherwise.

Once the root password has been set, thereafter you must supply that password when you connect to the server as root.

You may wish to leave the root password blank so that you don't need to specify it while you perform additional setup or testing. However, be sure to set it before using your installation for any real production work.

See the scripts/mysql_install_db script to see how it sets up the default privileges. You can use this as a basis to see how to add other users.

If you want the initial privileges to be different than those just described above, you can modify mysql_install_db before you run it.

To re-create the grant tables completely, remove all the `.frm', `.MYI', and `.MYD' files in the directory containing the mysql database. (This is the directory named `mysql' under the database directory, which is listed when you run mysqld --help.) Then run the mysql_install_db script, possibly after editing it first to have the privileges you want.

NOTE: For MySQL versions older than Version 3.22.10, you should NOT delete the `.frm' files. If you accidentally do this, you should copy them back from your MySQL distribution before running mysql_install_db.

4.3.5 Adding New Users to MySQL

You can add users two different ways: by using GRANT statements or by manipulating the MySQL grant tables directly. The preferred method is to use GRANT statements, because they are more concise and less error-prone. See section 4.3.1 GRANT and REVOKE Syntax.

There is also a lot of contributed programs like phpmyadmin that can be used to create and administrate users. See section D Contributed Programs.

The examples below show how to use the mysql client to set up new users. These examples assume that privileges are set up according to the defaults described in the previous section. This means that to make changes, you must be on the same machine where mysqld is running, you must connect as the MySQL root user, and the root user must have the insert privilege for the mysql database and the reload administrative privilege. Also, if you have changed the root user password, you must specify it for the mysql commands below.

You can add new users by issuing GRANT statements:

shell> mysql --user=root mysql
mysql> GRANT ALL PRIVILEGES ON *.* TO monty@localhost
           IDENTIFIED BY 'some_pass' WITH GRANT OPTION;
mysql> GRANT ALL PRIVILEGES ON *.* TO monty@"%"
           IDENTIFIED BY 'some_pass' WITH GRANT OPTION;
mysql> GRANT RELOAD,PROCESS ON *.* TO admin@localhost;
mysql> GRANT USAGE ON *.* TO dummy@localhost;

These GRANT statements set up three new users:

monty

A full superuser who can connect to the server from anywhere, but who must use a password 'some_pass' to do so. Note that we must issue GRANT statements for both monty@localhost and monty@"%". If we don't add the entry with localhost, the anonymous user entry for localhost that is created by mysql_install_db will take precedence when we connect from the local host, because it has a more specific Host field value and thus comes earlier in the user table sort order.

admin

A user who can connect from localhost without a password and who is granted the reload and process administrative privileges. This allows the user to execute the mysqladmin reload, mysqladmin refresh, and mysqladmin flush-* commands, as well as mysqladmin processlist . No database-related privileges are granted. (They can be granted later by issuing additional GRANT statements.)

dummy

A user who can connect without a password, but only from the local host. The global privileges are all set to 'N' -- the USAGE privilege type allows you to create a user with no privileges. It is assumed that you will grant database-specific privileges later.

You can also add the same user access information directly by issuing INSERT statements and then telling the server to reload the grant tables:

shell> mysql --user=root mysql
mysql> INSERT INTO user VALUES('localhost','monty',PASSWORD('some_pass'),
                'Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y');
mysql> INSERT INTO user VALUES('%','monty',PASSWORD('some_pass'),
                'Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y');
mysql> INSERT INTO user SET Host='localhost',User='admin',
                 Reload_priv='Y', Process_priv='Y';
mysql> INSERT INTO user (Host,User,Password)
                        VALUES('localhost','dummy','');
mysql> FLUSH PRIVILEGES;

Depending on your MySQL version, you may have to use a different number of 'Y' values above (versions prior to Version 3.22.11 had fewer privilege columns). For the admin user, the more readable extended INSERT syntax that is available starting with Version 3.22.11 is used.

Note that to set up a superuser, you need only create a user table entry with the privilege fields set to 'Y'. No db or host table entries are necessary.

The privilege columns in the user table were not set explicitly in the last INSERT statement (for the dummy user), so those columns are assigned the default value of 'N'. This is the same thing that GRANT USAGE does.

The following example adds a user custom who can connect from hosts localhost, server.domain, and whitehouse.gov. He wants to access the bankaccount database only from localhost, the expenses database only from whitehouse.gov, and the customer database from all three hosts. He wants to use the password stupid from all three hosts.

To set up this user's privileges using GRANT statements, run these commands:

shell> mysql --user=root mysql
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP
           ON bankaccount.*
           TO custom@localhost
           IDENTIFIED BY 'stupid';
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP
           ON expenses.*
           TO custom@whitehouse.gov
           IDENTIFIED BY 'stupid';
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP
           ON customer.*
           TO custom@'%'
           IDENTIFIED BY 'stupid';

The reason that we do to grant statements for the user 'custom' is that we want the give the user access to MySQL both from the local machine with Unix sockets and from the remote machine 'whitehouse.gov' over TCP/IP.

To set up the user's privileges by modifying the grant tables directly, run these commands (note the FLUSH PRIVILEGES at the end):

shell> mysql --user=root mysql
mysql> INSERT INTO user (Host,User,Password)
       VALUES('localhost','custom',PASSWORD('stupid'));
mysql> INSERT INTO user (Host,User,Password)
       VALUES('server.domain','custom',PASSWORD('stupid'));
mysql> INSERT INTO user (Host,User,Password)
       VALUES('whitehouse.gov','custom',PASSWORD('stupid'));
mysql> INSERT INTO db
       (Host,Db,User,Select_priv,Insert_priv,Update_priv,Delete_priv,
        Create_priv,Drop_priv)
       VALUES
       ('localhost','bankaccount','custom','Y','Y','Y','Y','Y','Y');
mysql> INSERT INTO db
       (Host,Db,User,Select_priv,Insert_priv,Update_priv,Delete_priv,
        Create_priv,Drop_priv)
       VALUES
       ('whitehouse.gov','expenses','custom','Y','Y','Y','Y','Y','Y');
mysql> INSERT INTO db
       (Host,Db,User,Select_priv,Insert_priv,Update_priv,Delete_priv,
        Create_priv,Drop_priv)
       VALUES('%','customer','custom','Y','Y','Y','Y','Y','Y');
mysql> FLUSH PRIVILEGES;

The first three INSERT statements add user table entries that allow user custom to connect from the various hosts with the given password, but grant no permissions to him (all privileges are set to the default value of 'N'). The next three INSERT statements add db table entries that grant privileges to custom for the bankaccount, expenses, and customer databases, but only when accessed from the proper hosts. As usual, when the grant tables are modified directly, the server must be told to reload them (with FLUSH PRIVILEGES) so that the privilege changes take effect.

If you want to give a specific user access from any machine in a given domain, you can issue a GRANT statement like the following:

mysql> GRANT ...
           ON *.*
           TO myusername@"%.mydomainname.com"
           IDENTIFIED BY 'mypassword';

To do the same thing by modifying the grant tables directly, do this:

mysql> INSERT INTO user VALUES ('%.mydomainname.com', 'myusername',
           PASSWORD('mypassword'),...);
mysql> FLUSH PRIVILEGES;

You can also use xmysqladmin, mysql_webadmin, and even xmysql to insert, change, and update values in the grant tables. You can find these utilities in the Contrib directory of the MySQL Website.

4.3.6 Setting Up Passwords

In most cases you should use GRANT to set up your users/passwords, so the following only applies for advanced users. See section 4.3.1 GRANT and REVOKE Syntax.

The examples in the preceding sections illustrate an important principle: when you store a non-empty password using INSERT or UPDATE statements, you must use the PASSWORD() function to encrypt it. This is because the user table stores passwords in encrypted form, not as plaintext. If you forget that fact, you are likely to attempt to set passwords like this:

shell> mysql -u root mysql
mysql> INSERT INTO user (Host,User,Password)
       VALUES('%','jeffrey','biscuit');
mysql> FLUSH PRIVILEGES;

The result is that the plaintext value 'biscuit' is stored as the password in the user table. When the user jeffrey attempts to connect to the server using this password, the mysql client encrypts it with PASSWORD(), generates an authentification vector based on encrypted password and a random number, obtained from server, and sends the result to the server. The server uses the password value in the user table (that is not encrypted value 'biscuit') to perform the same calculations, and compares results. The comparison fails and the server rejects the connection:

shell> mysql -u jeffrey -pbiscuit test
Access denied

Passwords must be encrypted when they are inserted in the user table, so the INSERT statement should have been specified like this instead:

mysql> INSERT INTO user (Host,User,Password)
       VALUES('%','jeffrey',PASSWORD('biscuit'));

You must also use the PASSWORD() function when you use SET PASSWORD statements:

mysql> SET PASSWORD FOR jeffrey@"%" = PASSWORD('biscuit');

If you set passwords using the GRANT ... IDENTIFIED BY statement or the mysqladmin password command, the PASSWORD() function is unnecessary. They both take care of encrypting the password for you, so you would specify a password of 'biscuit' like this:

mysql> GRANT USAGE ON *.* TO jeffrey@"%" IDENTIFIED BY 'biscuit';

or

shell> mysqladmin -u jeffrey password biscuit

NOTE: PASSWORD() does not perform password encryption in the same way that Unix passwords are encrypted. You should not assume that if your Unix password and your MySQL password are the same, that PASSWORD() will result in the same encrypted value as is stored in the Unix password file. See section 4.3.2 MySQL User Names and Passwords.

4.3.7 Keeping Your Password Secure

It is inadvisable to specify your password in a way that exposes it to discovery by other users. The methods you can use to specify your password when you run client programs are listed below, along with an assessment of the risks of each method:

• Never give a normal user access to the mysql.user table. Knowing the encrypted password for a user makes it possible to login as this user. The passwords are only scrambled so that one shouldn't be able to see the real password you used (if you happen to use a similar password with your other applications).
• Use a -pyour_pass or --password=your_pass option on the command line. This is convenient but insecure, because your password becomes visible to system status programs (such as ps) that may be invoked by other users to display command lines. (MySQL clients typically overwrite the command-line argument with zeroes during their initialization sequence, but there is still a brief interval during which the value is visible.)
• Use a -p or --password option (with no your_pass value specified). In this case, the client program solicits the password from the terminal:

  shell> mysql -u user_name -p
  Enter password: ********
  

  The `*' characters represent your password. It is more secure to enter your password this way than to specify it on the command line because it is not visible to other users. However, this method of entering a password is suitable only for programs that you run interactively. If you want to invoke a client from a script that runs non-interactively, there is no opportunity to enter the password from the terminal. On some systems, you may even find that the first line of your script is read and interpreted (incorrectly) as your password!
• Store your password in a configuration file. For example, you can list your password in the [client] section of the `.my.cnf' file in your home directory:

  [client]
  password=your_pass
  

  If you store your password in `.my.cnf', the file should not be group or world readable or writable. Make sure the file's access mode is 400 or 600. See section 4.1.2 my.cnf Option Files.
• You can store your password in the MYSQL_PWD environment variable, but this method must be considered extremely insecure and should not be used. Some versions of ps include an option to display the environment of running processes; your password will be in plain sight for all to see if you set MYSQL_PWD. Even on systems without such a version of ps, it is unwise to assume there is no other method to observe process environments. See section H Environment Variables.

All in all, the safest methods are to have the client program prompt for the password or to specify the password in a properly protected `.my.cnf' file.

4.4 Disaster Prevention and Recovery

4.4.1 Database Backups

Because MySQL tables are stored as files, it is easy to do a backup. To get a consistent backup, do a LOCK TABLES on the relevant tables followed by FLUSH TABLES for the tables. See section 6.7.2 LOCK TABLES/UNLOCK TABLES Syntax. See section 4.5.3 FLUSH Syntax. You only need a read lock; this allows other threads to continue to query the tables while you are making a copy of the files in the database directory. The FLUSH TABLE is needed to ensure that the all active index pages is written to disk before you start the backup.

If you want to make a SQL level backup of a table, you can use SELECT INTO OUTFILE or BACKUP TABLE. See section 6.4.1 SELECT Syntax. See section 4.4.2 BACKUP TABLE Syntax.

Another way to back up a database is to use the mysqldump program or the mysqlhotcopy script. See section 4.8.5 mysqldump, Dumping Table Structure and Data. See section 4.8.6 mysqlhotcopy, Copying MySQL Databases and Tables.

1. Do a full backup of your databases:

   shell> mysqldump --tab=/path/to/some/dir --opt --full
   
   or
   
   shell> mysqlhotcopy database /path/to/some/dir
   

   You can also simply copy all table files (`*.frm', `*.MYD', and `*.MYI' files) as long as the server isn't updating anything. The script mysqlhotcopy does use this method.
2. Stop mysqld if it's running, then start it with the --log-update[=file_name] option. See section 4.9.3 The Update Log. The update log file(s) provide you with the information you need to replicate changes to the database that are made subsequent to the point at which you executed mysqldump.

If you have to restore something, try to recover your tables using REPAIR TABLE or myisamchk -r first. That should work in 99.9% of all cases. If myisamchk fails, try the following procedure: (This will only work if you have started MySQL with --log-update. See section 4.9.3 The Update Log.):

1. Restore the original mysqldump backup.
2. Execute the following command to re-run the updates in the binary log:

   shell> mysqlbinlog hostname-bin.[0-9]* | mysql
   

   If you are using the update log you can use:

   shell> ls -1 -t -r hostname.[0-9]* | xargs cat | mysql
   

ls is used to get all the update log files in the right order.

You can also do selective backups with SELECT * INTO OUTFILE 'file_name' FROM tbl_name and restore with LOAD DATA INFILE 'file_name' REPLACE ... To avoid duplicate records, you need a PRIMARY KEY or a UNIQUE key in the table. The REPLACE keyword causes old records to be replaced with new ones when a new record duplicates an old record on a unique key value.

If you get performance problems in making backups on your system, you can solve this by setting up replication and do the backups on the slave instead of on the master. See section 4.10.1 Introduction.

If you are using a Veritas file system, you can do:

1. Execute in a client (perl ?) FLUSH TABLES WITH READ LOCK
2. Fork a shell or execute in another client mount vxfs snapshot.
3. Execute in the first client UNLOCK TABLES
4. Copy files from snapshot
5. Unmount snapshot

4.4.2 BACKUP TABLE Syntax

BACKUP TABLE tbl_name[,tbl_name...] TO '/path/to/backup/directory'

Make a copy of all the table files to the backup directory that are the minimum needed to restore it. Currenlty only works for MyISAM tables. For MyISAM table, copies .frm (definition) and .MYD (data) files. The index file can be rebuilt from those two.

Before using this command, please see See section 4.4.1 Database Backups.

During the backup, read lock will be held for each table, one at time, as they are being backed up. If you want to backup several tables as a snapshot, you must first issue LOCK TABLES obtaining a read lock for each table in the group.

The command returns a table with the following columns:

Note that BACKUP TABLE is only available in MySQL version 3.23.25 and later.

4.4.3 RESTORE TABLE Syntax

RESTORE TABLE tbl_name[,tbl_name...] FROM '/path/to/backup/directory'

Restores the table(s) from the backup that was made with BACKUP TABLE. Existing tables will not be overwritten - if you try to restore over an existing table, you will get an error. Restore will take longer than BACKUP due to the need to rebuilt the index. The more keys you have, the longer it is going to take. Just as BACKUP TABLE, currently only works of MyISAM tables.

The command returns a table with the following columns:

4.4.4 CHECK TABLE Syntax

CHECK TABLE tbl_name[,tbl_name...] [option [option...]]

option = QUICK | FAST | MEDIUM | EXTENDED | CHANGED

CHECK TABLE only works on MyISAM tables. On MyISAM tables it's the same thing as running myisamchk -m table_name on the table.

If you don't specify any option MEDIUM is used.

Checks the table(s) for errors. For MyISAM tables the key statistics is updated. The command returns a table with the following columns:

Note that you can get many rows of information for each checked table. The last row will be of Msg_type status and should normally be OK. If you don't get OK, or Not checked you should normally run a repair of the table. See section 4.4.6 Using myisamchk for Table Maintenance and Crash Recovery. Not checked means that the table the given TYPE told MySQL that there wasn't any need to check the table.

The different check types stand for the following:

For dynamic sized MyISAM tables a started check will always do a MEDIUM check. For static size rows we skip the row scan for QUICK and FAST as the rows are very seldom corrupted.

You can combine check options as in:

CHECK TABLE test_table FAST QUICK;

Which only would do a quick check on the table if it wasn't closed properly.

NOTE: that in some case CHECK TABLE will change the table! This happens if the table is marked as 'corrupted' or 'not closed properly' but CHECK TABLE didn't find any problems in the table. In this case CHECK TABLE will mark the table as ok.

If a table is corrupted, then it's most likely that the problem is in the indexes and not in the data part. All of the above check types checks the indexes throughly and should thus find most errors.

If you just want to check a table that you assume is ok, you should use no check options or the QUICK option. The later should be used when you are in a hurry and can take the very small risk that QUICK didn't find an error in the data file (In most cases MySQL should find, under normal usage, any error in the data file. If this happens then the table will be marked as 'corrupted', in which case the table can't be used until it's repaired).

FAST and CHANGED are mostly intended to be used from a script (for example to be executed from cron) if you want to check your table from time to time. In most cases you FAST is to be prefered over CHANGED. (The only case when it isn't is when you suspect a bug you have found a bug in the MyISAM code.).

EXTENDED is only to be used after you have run a normal check but still get strange errors from a table when MySQL tries to update a row or find a row by key (this is VERY unlikely to happen if a normal check has succeeded!).

Some things reported by check table, can't be corrected automatically:

• Found row where the auto_increment column has the value 0. This means that you have in the table a row where the auto_increment index column contains the value 0. (It's possible to create a row where the auto_increment column is 0 by explicitely setting the column to 0 with an UPDATE statement) This isn't an error in itself, but could cause trouble if you decide to dump the table and restore it or do an ALTER TABLE on the table. In this case the auto_increment column will change value, according to the rules of auto_increment columns, which could cause problems like a duplicate key error. To get rid of the warning, just execute an UPDATE statement to set the column to some other value than 0.

4.4.5 REPAIR TABLE Syntax

REPAIR TABLE tbl_name[,tbl_name...] [QUICK] [EXTENDED]

REPAIR TABLE only works on MyISAM tables and is the same as running myisamchk -r table_name on the table.

Normally you should never have to run this command, but if disaster strikes you are very likely to get back all your data from a MyISAM table with REPAIR TABLE. If your tables get corrupted a lot you should try to find the reason for this! See section A.4.1 What To Do If MySQL Keeps Crashing. See section 7.1.3 MyISAM table problems..

REPAIR TABLE repairs a possible corrupted table. The command returns a table with the following columns:

Note that you can get many rows of information for each repaired table. The last one row will be of Msg_type status and should normally be OK. If you don't get OK, you should try repairing the table with myisamchk -o, as REPAIR TABLE does not yet implement all the options of myisamchk. In the near future, we will make it more flexible.

If QUICK is given then MySQL will try to do a REPAIR of only the index tree.

If you use EXTENDED then MySQL will create the index row by row instead of creating one index at a time with sorting; This may be better than sorting on fixed-length keys if you have long char() keys that compress very good.

4.4.6 Using myisamchk for Table Maintenance and Crash Recovery

Starting with MySQL Version 3.23.13, you can check MyISAM tables with the CHECK TABLE command. See section 4.4.4 CHECK TABLE Syntax. You can repair tables with the REPAIR TABLE command. See section 4.4.5 REPAIR TABLE Syntax.

To check/repair MyISAM tables (.MYI and .MYD) you should use the myisamchk utility. To check/repair ISAM tables (.ISM and .ISD) you should use the isamchk utility. See section 7 MySQL Table Types.

In the following text we will talk about myisamchk, but everything also applies to the old isamchk.

You can use the myisamchk utility to get information about your database tables, check and repair them, or optimize them. The following sections describe how to invoke myisamchk (including a description of its options), how to set up a table maintenance schedule, and how to use myisamchk to perform its various functions.

You can, in most cases, also use the command OPTIMIZE TABLES to optimize and repair tables, but this is not as fast or reliable (in case of real fatal errors) as myisamchk. On the other hand, OPTIMIZE TABLE is easier to use and you don't have to worry about flushing tables. See section 4.5.1 OPTIMIZE TABLE Syntax.

Even that the repair in myisamchk is quite secure, it's always a good idea to make a backup BEFORE doing a repair (or anything that could make a lot of changes to a table)

4.4.6.1 myisamchk Invocation Syntax

myisamchk is invoked like this:

shell> myisamchk [options] tbl_name

The options specify what you want myisamchk to do. They are described below. (You can also get a list of options by invoking myisamchk --help.) With no options, myisamchk simply checks your table. To get more information or to tell myisamchk to take corrective action, specify options as described below and in the following sections.

tbl_name is the database table you want to check/repair. If you run myisamchk somewhere other than in the database directory, you must specify the path to the file, because myisamchk has no idea where your database is located. Actually, myisamchk doesn't care whether or not the files you are working on are located in a database directory; you can copy the files that correspond to a database table into another location and perform recovery operations on them there.

You can name several tables on the myisamchk command line if you wish. You can also specify a name as an index file name (with the `.MYI' suffix), which allows you to specify all tables in a directory by using the pattern `*.MYI'. For example, if you are in a database directory, you can check all the tables in the directory like this:

shell> myisamchk *.MYI

If you are not in the database directory, you can check all the tables there by specifying the path to the directory:

shell> myisamchk /path/to/database_dir/*.MYI

You can even check all tables in all databases by specifying a wild card with the path to the MySQL data directory:

shell> myisamchk /path/to/datadir/*/*.MYI

The recommended way to quickly check all tables is:

myisamchk --silent --fast /path/to/datadir/*/*.MYI
isamchk --silent /path/to/datadir/*/*.ISM

If you want to check all tables and repair all tables that are corrupted, you can use the following line:

myisamchk --silent --force --fast --update-state -O key_buffer=64M -O sort_buffer=64M -O read_buffer=1M -O write_buffer=1M /path/to/datadir/*/*.MYI
isamchk --silent --force -O key_buffer=64M -O sort_buffer=64M -O read_buffer=1M -O write_buffer=1M /path/to/datadir/*/*.ISM

The above assumes that you have more than 64 M free.

Note that if you get an error like:

myisamchk: warning: 1 clients is using or hasn't closed the table properly

This means that you are trying to check a table that has been updated by the another program (like the mysqld server) that hasn't yet closed the file or that has died without closing the file properly.

If you mysqld is running, you must force a sync/close of all tables with FLUSH TABLES and ensure that no one is using the tables while you are running myisamchk. In MySQL Version 3.23 the easiest way to avoid this problem is to use CHECK TABLE instead of myisamchk to check tables.

4.4.6.2 General Options for myisamchk

myisamchk supports the following options.

-# or --debug=debug_options

Output debug log. The debug_options string often is 'd:t:o,filename'.

-? or --help

Display a help message and exit.

-O var=option, --set-variable var=option

Set the value of a variable. The possible variables and their default values for myisamchk can be examined with myisamchk --help:

key_buffer_size	523264
read_buffer_size	262136
write_buffer_size	262136
sort_buffer_size	2097144
sort_key_blocks	16
decode_bits	9

sort_buffer_size is used when the keys are repaired by sorting keys, which is the normal case when you use --recover. key_buffer_size is used when you are checking the table with --extended-check or when the keys are repaired by inserting key row by row in to the table (like when doing normal inserts). Repairing through the key buffer is used in the following cases:

• If you use --safe-recover.
• If the temporary files needed to sort the keys would be more than twice as big as when creating the key file directly. This is often the case when you have big CHAR, VARCHAR or TEXT keys as the sort needs to store the whole keys during sorting. If you have lots of temporary space and you can force myisamchk to repair by sorting you can use the --sort-recover option.

Reparing through the key buffer takes much less disk space than using sorting, but is also much slower. If you want a faster repair, set the above variables to about 1/4 of your available memory. You can set both variables to big values, as only one of the above buffers will be used at a time.

-s or --silent

Silent mode. Write output only when errors occur. You can use -s twice (-ss) to make myisamchk very silent.

-v or --verbose

Verbose mode. Print more information. This can be used with -d and -e. Use -v multiple times (-vv, -vvv) for more verbosity!

-V or --version

Print the myisamchk version and exit.

-w or, --wait

Instead of giving an error if the table is locked, wait until the table is unlocked before continuing. Note that if you are running mysqld on the table with --skip-locking, the table can only be locked by another myisamchk command.

4.4.6.3 Check Options for myisamchk

-c or --check

Check table for errors. This is the default operation if you are not giving myisamchk any options that override this.

-e or --extend-check

Check the table VERY thoroughly (which is quite slow if you have many indexes). This option should only be used in extreme cases. Normally, myisamchk or myisamchk --medium-check should, in most cases, be able to find out if there are any errors in the table. If you are using --extended-check and have much memory, you should increase the value of key_buffer_size a lot!

-F or --fast

Check only tables that haven't been closed properly.

-C or --check-only-changed

Check only tables that have changed since the last check.

-f or --force

Restart myisamchk with -r (repair) on the table, if myisamchk finds any errors in the table.

-i or --information

Print informational statistics about the table that is checked.

-m or --medium-check

Faster than extended-check, but only finds 99.99% of all errors. Should, however, be good enough for most cases.

-U or --update-state

Store in the `.MYI' file when the table was checked and if the table crashed. This should be used to get full benefit of the --check-only-changed option, but you shouldn't use this option if the mysqld server is using the table and you are running mysqld with --skip-locking.

-T or --read-only

Don't mark table as checked. This is useful if you use myisamchk to check a table that is in use by some other application that doesn't use locking (like mysqld --skip-locking).

4.4.6.4 Repair Options for myisamchk

The following options are used if you start myisamchk with -r or -o:

-D # or --data-file-length=#

Max length of data file (when re-creating data file when it's 'full').

-e or --extend-check

Try to recover every possible row from the data file. Normally this will also find a lot of garbage rows. Don't use this option if you are not totally desperate.

-f or --force

Overwrite old temporary files (table_name.TMD) instead of aborting.

-k # or keys-used=#

If you are using ISAM, tells the ISAM table handler to update only the first # indexes. If you are using MyISAM, tells which keys to use, where each binary bit stands for one key (first key is bit 0). This can be used to get faster inserts! Deactivated indexes can be reactivated by using myisamchk -r. keys.

-l or --no-symlinks

Do not follow symbolic links. Normally myisamchk repairs the table a symlink points at. This option doesn't exist in MySQL 4.0, as MySQL 4.0 will not remove symlinks during repair.

-r or --recover

Can fix almost anything except unique keys that aren't unique (which is an extremely unlikely error with ISAM/MyISAM tables). If you want to recover a table, this is the option to try first. Only if myisamchk reports that the table can't be recovered by -r, you should then try -o. (Note that in the unlikely case that -r fails, the data file is still intact.) If you have lots of memory, you should increase the size of sort_buffer_size!

-o or --safe-recover

Uses an old recovery method (reads through all rows in order and updates all index trees based on the found rows); this is a magnitude slower than -r, but can handle a couple of very unlikely cases that -r cannot handle. This recovery method also uses much less disk space than -r. Normally one should always first repair with -r, and only if this fails use -o. If you have lots of memory, you should increase the size of key_buffer_size!

-n or --sort-recover

Force myisamchk to use sorting to resolve the keys even if the temporary files should be very big. This will not have any effect if you have fulltext keys in the table.

--character-sets-dir=...

Directory where character sets are stored.

--set-character-set=name

Change the character set used by the index

.t or --tmpdir=path

Path for storing temporary files. If this is not set, myisamchk will use the environment variable TMPDIR for this.

-q or --quick

Faster repair by not modifying the data file. One can give a second -q to force myisamchk to modify the original datafile in case of duplicate keys

-u or --unpack

Unpack file packed with myisampack.

4.4.6.5 Other Options for myisamchk

Other actions that myisamchk can do, besides repair and check tables:

-a or --analyze

Analyze the distribution of keys. This improves join performance by enabling the join optimizer to better choose in which order it should join the tables and which keys it should use: myisamchk --describe --verbose table_name' or using SHOW KEYS in MySQL.

-d or --description

Prints some information about table.

-A or --set-auto-increment[=value]

Force auto_increment to start at this or higher value. If no value is given, then sets the next auto_increment value to the highest used value for the auto key + 1.

-S or --sort-index

Sort the index tree blocks in high-low order. This will optimize seeks and will make table scanning by key faster.

-R or --sort-records=#

Sorts records according to an index. This makes your data much more localized and may speed up ranged SELECT and ORDER BY operations on this index. (It may be VERY slow to do a sort the first time!) To find out a table's index numbers, use SHOW INDEX, which shows a table's indexes in the same order that myisamchk sees them. Indexes are numbered beginning with 1.

4.4.6.6 myisamchk Memory Usage

Memory allocation is important when you run myisamchk. myisamchk uses no more memory than you specify with the -O options. If you are going to use myisamchk on very large files, you should first decide how much memory you want it to use. The default is to use only about 3M to fix things. By using larger values, you can get myisamchk to operate faster. For example, if you have more than 32M RAM, you could use options such as these (in addition to any other options you might specify):

shell> myisamchk -O sort=16M -O key=16M -O read=1M -O write=1M ...

Using -O sort=16M should probably be enough for most cases.

Be aware that myisamchk uses temporary files in TMPDIR. If TMPDIR points to a memory file system, you may easily get out of memory errors. If this happens, set TMPDIR to point at some directory with more space and restart myisamchk.

When repairing, myisamchk will also need a lot of disk space:

• Double the size of the record file (the original one and a copy). This space is not needed if one does a repair with --quick, as in this case only the index file will be re-created. This space is needed on the same disk as the original record file!
• Space for the new index file that replaces the old one. The old index file is truncated at start, so one usually ignore this space. This space is needed on the same disk as the original index file!
• When using --recover or --sort-recover (but not when using --safe-recover, you will need space for a sort buffer for: (largest_key + row_pointer_length)*number_of_rows * 2. You can check the length of the keys and the row_pointer_length with myisamchk -dv table. This space is allocated on the temporary disk (specified by TMPDIR or --tmpdir=#).

If you have a problem with disk space during repair, you can try to use --safe-recover instead of --recover.

4.4.6.7 Using myisamchk for Crash Recovery

If you run mysqld with --skip-locking (which is the default on some systems, like Linux), you can't reliably use myisamchk to check a table when mysqld is using the same table. If you can be sure that no one is accessing the tables through mysqld while you run myisamchk, you only have to do mysqladmin flush-tables before you start checking the tables. If you can't guarantee the above, then you must take down mysqld while you check the tables. If you run myisamchk while mysqld is updating the tables, you may get a warning that a table is corrupt even if it isn't.

If you are not using --skip-locking, you can use myisamchk to check tables at any time. While you do this, all clients that try to update the table will wait until myisamchk is ready before continuing.

If you use myisamchk to repair or optimize tables, you MUST always ensure that the mysqld server is not using the table (this also applies if you are using --skip-locking). If you don't take down mysqld you should at least do a mysqladmin flush-tables before you run myisamchk.

This chapter describes how to check for and deal with data corruption in MySQL databases. If your tables get corrupted a lot you should try to find the reason for this! See section A.4.1 What To Do If MySQL Keeps Crashing.

The MyISAM table section contains reason for why a table could be corrupted. See section 7.1.3 MyISAM table problems..

When performing crash recovery, it is important to understand that each table tbl_name in a database corresponds to three files in the database directory:

Each of these three file types is subject to corruption in various ways, but problems occur most often in data files and index files.

myisamchk works by creating a copy of the `.MYD' (data) file row by row. It ends the repair stage by removing the old `.MYD' file and renaming the new file to the original file name. If you use --quick, myisamchk does not create a temporary `.MYD' file, but instead assumes that the `.MYD' file is correct and only generates a new index file without touching the `.MYD' file. This is safe, because myisamchk automatically detects if the `.MYD' file is corrupt and aborts the repair in this case. You can also give two --quick options to myisamchk. In this case, myisamchk does not abort on some errors (like duplicate key) but instead tries to resolve them by modifying the `.MYD' file. Normally the use of two --quick options is useful only if you have too little free disk space to perform a normal repair. In this case you should at least make a backup before running myisamchk.

4.4.6.8 How to Check Tables for Errors

To check a MyISAM table, use the following commands:

myisamchk tbl_name

This finds 99.99% of all errors. What it can't find is corruption that involves ONLY the data file (which is very unusual). If you want to check a table, you should normally run myisamchk without options or with either the -s or --silent option.

myisamchk -m tbl_name

This finds 99.999% of all errors. It checks first all index entries for errors and then it reads through all rows. It calculates a checksum for all keys in the rows and verifies that they checksum matches the checksum for the keys in the index tree.

myisamchk -e tbl_name

This does a complete and thorough check of all data (-e means ``extended check''). It does a check-read of every key for each row to verify that they indeed point to the correct row. This may take a LONG time on a big table with many keys. myisamchk will normally stop after the first error it finds. If you want to obtain more information, you can add the --verbose (-v) option. This causes myisamchk to keep going, up through a maximum of 20 errors. In normal usage, a simple myisamchk (with no arguments other than the table name) is sufficient.

myisamchk -e -i tbl_name

Like the previous command, but the -i option tells myisamchk to print some informational statistics, too.

4.4.6.9 How to Repair Tables

In the following section we only talk about using myisamchk on MyISAM tables (extensions .MYI and .MYD). If you are using ISAM tables (extensions .ISM and .ISD), you should use isamchk instead.

Starting with MySQL Version 3.23.14, you can repair MyISAM tables with the REPAIR TABLE command. See section 4.4.5 REPAIR TABLE Syntax.

The symptoms of a corrupted table include queries that abort unexpectedly and observable errors such as these:

• `tbl_name.frm' is locked against change
• Can't find file `tbl_name.MYI' (Errcode: ###)
• Unexpected end of file
• Record file is crashed
• Got error ### from table handler To get more information about the error you can run perror ###. Here is the most common errors that indicates a problem with the table:

  shell> perror 126 127 132 134 135 136 141 144 145
  126 = Index file is crashed / Wrong file format
  127 = Record-file is crashed
  132 = Old database file
  134 = Record was already deleted (or record file crashed)
  135 = No more room in record file
  136 = No more room in index file
  141 = Duplicate unique key or constraint on write or update
  144 = Table is crashed and last repair failed
  145 = Table was marked as crashed and should be repaired
  

  Note that error 135, no more room in record file, is not an error that can be fixed by a simple repair. In this case you have to do:

  ALTER TABLE table MAX_ROWS=xxx AVG_ROW_LENGTH=yyy;
  

In the other cases, you must repair your tables. myisamchk can usually detect and fix most things that go wrong.

The repair process involves up to four stages, described below. Before you begin, you should cd to the database directory and check the permissions of the table files. Make sure they are readable by the Unix user that mysqld runs as (and to you, because you need to access the files you are checking). If it turns out you need to modify files, they must also be writable by you.

If you are using MySQL Version 3.23.16 and above, you can (and should) use the CHECK and REPAIR commands to check and repair MyISAM tables. See section 4.4.4 CHECK TABLE Syntax. See section 4.4.5 REPAIR TABLE Syntax.

The manual section about table maintenance includes the options to isamchk/myisamchk. See section 4.4.6 Using myisamchk for Table Maintenance and Crash Recovery.

The following section is for the cases where the above command fails or if you want to use the extended features that isamchk/myisamchk provides.

If you are going to repair a table from the command line, you must first take down the mysqld server. Note that when you do mysqladmin shutdown on a remote server, the mysqld server will still be alive for a while after mysqladmin returns, until all queries are stopped and all keys have been flushed to disk.

Stage 1: Checking your tables

Run myisamchk *.MYI or myisamchk -e *.MYI if you have more time. Use the -s (silent) option to suppress unnecessary information.

If the mysqld server is done you should use the --update option to tell myisamchk to mark the table as 'checked'.

You have to repair only those tables for which myisamchk announces an error. For such tables, proceed to Stage 2.

If you get weird errors when checking (such as out of memory errors), or if myisamchk crashes, go to Stage 3.

Stage 2: Easy safe repair

NOTE: If you want repairing to go much faster, you should add: -O sort_buffer=# -O key_buffer=# (where # is about 1/4 of the available memory) to all isamchk/myisamchk commands.

First, try myisamchk -r -q tbl_name (-r -q means ``quick recovery mode''). This will attempt to repair the index file without touching the data file. If the data file contains everything that it should and the delete links point at the correct locations within the data file, this should work, and the table is fixed. Start repairing the next table. Otherwise, use the following procedure:

1. Make a backup of the data file before continuing.
2. Use myisamchk -r tbl_name (-r means ``recovery mode''). This will remove incorrect records and deleted records from the data file and reconstruct the index file.
3. If the preceding step fails, use myisamchk --safe-recover tbl_name. Safe recovery mode uses an old recovery method that handles a few cases that regular recovery mode doesn't (but is slower).

If you get weird errors when repairing (such as out of memory errors), or if myisamchk crashes, go to Stage 3.

Stage 3: Difficult repair

You should only reach this stage if the first 16K block in the index file is destroyed or contains incorrect information, or if the index file is missing. In this case, it's necessary to create a new index file. Do so as follows:

1. Move the data file to some safe place.
2. Use the table description file to create new (empty) data and index files:

   shell> mysql db_name
   mysql> SET AUTOCOMMIT=1;
   mysql> TRUNCATE TABLE table_name;
   mysql> quit
   

   If your SQL version doesn't have TRUNCATE TABLE, use DELETE FROM table_name instead.
3. Copy the old data file back onto the newly created data file. (Don't just move the old file back onto the new file; you want to retain a copy in case something goes wrong.)

Go back to Stage 2. myisamchk -r -q should work now. (This shouldn't be an endless loop.)

Stage 4: Very difficult repair

You should reach this stage only if the description file has also crashed. That should never happen, because the description file isn't changed after the table is created:

1. Restore the description file from a backup and go back to Stage 3. You can also restore the index file and go back to Stage 2. In the latter case, you should start with myisamchk -r.
2. If you don't have a backup but know exactly how the table was created, create a copy of the table in another database. Remove the new data file, then move the description and index files from the other database to your crashed database. This gives you new description and index files, but leaves the data file alone. Go back to Stage 2 and attempt to reconstruct the index file.

4.4.6.10 Table Optimization

To coalesce fragmented records and eliminate wasted space resulting from deleting or updating records, run myisamchk in recovery mode:

shell> myisamchk -r tbl_name

You can optimize a table in the same way using the SQL OPTIMIZE TABLE statement. OPTIMIZE TABLE does a repair of the table, a key analyzes and also sorts the index tree to give faster key lookups. There is also no possibility of unwanted interaction between a utility and the server, because the server does all the work when you use OPTIMIZE TABLE. See section 4.5.1 OPTIMIZE TABLE Syntax.

myisamchk also has a number of other options you can use to improve the performance of a table:

-S, --sort-index

-R index_num, --sort-records=index_num

-a, --analyze

For a full description of the option. See section 4.4.6.1 myisamchk Invocation Syntax.

4.4.7 Setting Up a Table Maintenance Regimen

Starting with MySQL Version 3.23.13, you can check MyISAM tables with the CHECK TABLE command. See section 4.4.4 CHECK TABLE Syntax. You can repair tables with the REPAIR TABLE command. See section 4.4.5 REPAIR TABLE Syntax.

It is a good idea to perform table checks on a regular basis rather than waiting for problems to occur. For maintenance purposes, you can use myisamchk -s to check tables. The -s option (short for --silent) causes myisamchk to run in silent mode, printing messages only when errors occur.

It's also a good idea to check tables when the server starts up. For example, whenever the machine has done a reboot in the middle of an update, you usually need to check all the tables that could have been affected. (This is an ``expected crashed table''.) You could add a test to safe_mysqld that runs myisamchk to check all tables that have been modified during the last 24 hours if there is an old `.pid' (process ID) file left after a reboot. (The `.pid' file is created by mysqld when it starts up and removed when it terminates normally. The presence of a `.pid' file at system startup time indicates that mysqld terminated abnormally.)

An even better test would be to check any table whose last-modified time is more recent than that of the `.pid' file.

You should also check your tables regularly during normal system operation. At MySQL AB, we run a cron job to check all our important tables once a week, using a line like this in a `crontab' file:

35 0 * * 0 /path/to/myisamchk --fast --silent /path/to/datadir/*/*.MYI

This prints out information about crashed tables so we can examine and repair them when needed.

As we haven't had any unexpectedly crashed tables (tables that become corrupted for reasons other than hardware trouble) for a couple of years now (this is really true), once a week is more than enough for us.

We recommend that to start with, you execute myisamchk -s each night on all tables that have been updated during the last 24 hours, until you come to trust MySQL as much as we do.

Normally you don't need to maintain MySQL tables that much. If you are changing tables with dynamic size rows (tables with VARCHAR, BLOB or TEXT columns) or have tables with many deleted rows you may want to from time to time (once a month?) defragment/reclaim space from the tables.

You can do this by using OPTIMIZE TABLE on the tables in question or if you can take the mysqld server down for a while do:

isamchk -r --silent --sort-index -O sort_buffer_size=16M */*.ISM
myisamchk -r --silent --sort-index  -O sort_buffer_size=16M */*.MYI

4.4.8 Getting Information About a Table

To get a description of a table or statistics about it, use the commands shown below. We explain some of the information in more detail later:

myisamchk -d tbl_name

Runs myisamchk in ``describe mode'' to produce a description of your table. If you start the MySQL server using the --skip-locking option, myisamchk may report an error for a table that is updated while it runs. However, because myisamchk doesn't change the table in describe mode, there isn't any risk of destroying data.

myisamchk -d -v tbl_name

To produce more information about what myisamchk is doing, add -v to tell it to run in verbose mode.

myisamchk -eis tbl_name

Shows only the most important information from a table. It is slow because it must read the whole table.

myisamchk -eiv tbl_name

This is like -eis, but tells you what is being done.

Example of myisamchk -d output:

MyISAM file:     company.MYI
Record format:   Fixed length
Data records:    1403698  Deleted blocks:         0
Recordlength:    226

table description:
Key Start Len Index   Type
1   2     8   unique  double
2   15    10  multip. text packed stripped
3   219   8   multip. double
4   63    10  multip. text packed stripped
5   167   2   multip. unsigned short
6   177   4   multip. unsigned long
7   155   4   multip. text
8   138   4   multip. unsigned long
9   177   4   multip. unsigned long
    193   1           text

Example of myisamchk -d -v output:

MyISAM file:         company
Record format:       Fixed length
File-version:        1
Creation time:       1999-10-30 12:12:51
Recover time:        1999-10-31 19:13:01
Status:              checked
Data records:           1403698  Deleted blocks:              0
Datafile parts:         1403698  Deleted data:                0
Datafilepointer (bytes):      3  Keyfile pointer (bytes):     3
Max datafile length: 3791650815  Max keyfile length: 4294967294
Recordlength:               226

table description:
Key Start Len Index   Type                  Rec/key     Root Blocksize
1   2     8   unique  double                      1 15845376      1024
2   15    10  multip. text packed stripped        2 25062400      1024
3   219   8   multip. double                     73 40907776      1024
4   63    10  multip. text packed stripped        5 48097280      1024
5   167   2   multip. unsigned short           4840 55200768      1024
6   177   4   multip. unsigned long            1346 65145856      1024
7   155   4   multip. text                     4995 75090944      1024
8   138   4   multip. unsigned long              87 85036032      1024
9   177   4   multip. unsigned long             178 96481280      1024
    193   1           text

Example of myisamchk -eis output:

Checking MyISAM file: company
Key:  1:  Keyblocks used:  97%  Packed:    0%  Max levels:  4
Key:  2:  Keyblocks used:  98%  Packed:   50%  Max levels:  4
Key:  3:  Keyblocks used:  97%  Packed:    0%  Max levels:  4
Key:  4:  Keyblocks used:  99%  Packed:   60%  Max levels:  3
Key:  5:  Keyblocks used:  99%  Packed:    0%  Max levels:  3
Key:  6:  Keyblocks used:  99%  Packed:    0%  Max levels:  3
Key:  7:  Keyblocks used:  99%  Packed:    0%  Max levels:  3
Key:  8:  Keyblocks used:  99%  Packed:    0%  Max levels:  3
Key:  9:  Keyblocks used:  98%  Packed:    0%  Max levels:  4
Total:    Keyblocks used:  98%  Packed:   17%

Records:          1403698    M.recordlength:     226   Packed:             0%
Recordspace used:     100%   Empty space:          0%  Blocks/Record:   1.00
Record blocks:    1403698    Delete blocks:        0
Recorddata:     317235748    Deleted data:         0
Lost space:             0    Linkdata:             0

User time 1626.51, System time 232.36
Maximum resident set size 0, Integral resident set size 0
Non physical pagefaults 0, Physical pagefaults 627, Swaps 0
Blocks in 0 out 0, Messages in 0 out 0, Signals 0
Voluntary context switches 639, Involuntary context switches 28966

Example of myisamchk -eiv output:

Checking MyISAM file: company
Data records: 1403698   Deleted blocks:       0
- check file-size
- check delete-chain
block_size 1024:
index  1:
index  2:
index  3:
index  4:
index  5:
index  6:
index  7:
index  8:
index  9:
No recordlinks
- check index reference
- check data record references index: 1
Key:  1:  Keyblocks used:  97%  Packed:    0%  Max levels:  4
- check data record references index: 2
Key:  2:  Keyblocks used:  98%  Packed:   50%  Max levels:  4
- check data record references index: 3
Key:  3:  Keyblocks used:  97%  Packed:    0%  Max levels:  4
- check data record references index: 4
Key:  4:  Keyblocks used:  99%  Packed:   60%  Max levels:  3
- check data record references index: 5
Key:  5:  Keyblocks used:  99%  Packed:    0%  Max levels:  3
- check data record references index: 6
Key:  6:  Keyblocks used:  99%  Packed:    0%  Max levels:  3
- check data record references index: 7
Key:  7:  Keyblocks used:  99%  Packed:    0%  Max levels:  3
- check data record references index: 8
Key:  8:  Keyblocks used:  99%  Packed:    0%  Max levels:  3
- check data record references index: 9
Key:  9:  Keyblocks used:  98%  Packed:    0%  Max levels:  4
Total:    Keyblocks used:   9%  Packed:   17%

- check records and index references
[LOTS OF ROW NUMBERS DELETED]

Records:          1403698    M.recordlength:     226   Packed:             0%
Recordspace used:     100%   Empty space:          0%  Blocks/Record:   1.00
Record blocks:    1403698    Delete blocks:        0
Recorddata:     317235748    Deleted data:         0
Lost space:             0    Linkdata:             0

User time 1639.63, System time 251.61
Maximum resident set size 0, Integral resident set size 0
Non physical pagefaults 0, Physical pagefaults 10580, Swaps 0
Blocks in 4 out 0, Messages in 0 out 0, Signals 0
Voluntary context switches 10604, Involuntary context switches 122798

Here are the sizes of the data and index files for the table used in the preceding examples:

-rw-rw-r--   1 monty    tcx     317235748 Jan 12 17:30 company.MYD
-rw-rw-r--   1 davida   tcx      96482304 Jan 12 18:35 company.MYM

Explanations for the types of information myisamchk produces are given below. The ``keyfile'' is the index file. ``Record'' and ``row'' are synonymous:

ISAM file

Name of the ISAM (index) file.

Isam-version

Version of ISAM format. Currently always 2.

Creation time

When the data file was created.

Recover time

When the index/data file was last reconstructed.

Data records

How many records are in the table.

Deleted blocks

How many deleted blocks still have reserved space. You can optimize your table to minimize this space. See section 4.4.6.10 Table Optimization.

Datafile: Parts

For dynamic record format, this indicates how many data blocks there are. For an optimized table without fragmented records, this is the same as Data records.

Deleted data

How many bytes of non-reclaimed deleted data there are. You can optimize your table to minimize this space. See section 4.4.6.10 Table Optimization.

Datafile pointer

The size of the data file pointer, in bytes. It is usually 2, 3, 4, or 5 bytes. Most tables manage with 2 bytes, but this cannot be controlled from MySQL yet. For fixed tables, this is a record address. For dynamic tables, this is a byte address.

Keyfile pointer

The size of the index file pointer, in bytes. It is usually 1, 2, or 3 bytes. Most tables manage with 2 bytes, but this is calculated automatically by MySQL. It is always a block address.

Max datafile length

How long the table's data file (.MYD file) can become, in bytes.

Max keyfile length

How long the table's key file (.MYI file) can become, in bytes.

Recordlength

How much space each record takes, in bytes.

Record format

The format used to store table rows. The examples shown above use Fixed length. Other possible values are Compressed and Packed.

table description

A list of all keys in the table. For each key, some low-level information is presented:

Key

This key's number.

Start

Where in the record this index part starts.

Len

How long this index part is. For packed numbers, this should always be the full length of the column. For strings, it may be shorter than the full length of the indexed column, because you can index a prefix of a string column.

Index

unique or multip. (multiple). Indicates whether or not one value can exist multiple times in this index.

Type

What data-type this index part has. This is an ISAM data-type with the options packed, stripped or empty.

Root

Address of the root index block.

Blocksize

The size of each index block. By default this is 1024, but the value may be changed at compile time.

Rec/key

This is a statistical value used by the optimizer. It tells how many records there are per value for this key. A unique key always has a value of 1. This may be updated after a table is loaded (or greatly changed) with myisamchk -a. If this is not updated at all, a default value of 30 is given.

In the first example above, the 9th key is a multi-part key with two parts.

Keyblocks used

What percentage of the keyblocks are used. Because the table used in the examples had just been reorganized with myisamchk, the values are very high (very near the theoretical maximum).

Packed

MySQL tries to pack keys with a common suffix. This can only be used for CHAR/VARCHAR/DECIMAL keys. For long strings like names, this can significantly reduce the space used. In the third example above, the 4th key is 10 characters long and a 60% reduction in space is achieved.

Max levels

How deep the B-tree for this key is. Large tables with long keys get high values.

Records

How many rows are in the table.

M.recordlength

The average record length. For tables with fixed-length records, this is the exact record length.

Packed

MySQL strips spaces from the end of strings. The Packed value indicates the percentage of savings achieved by doing this.

Recordspace used

What percentage of the data file is used.

Empty space

What percentage of the data file is unused.

Blocks/Record

Average number of blocks per record (that is, how many links a fragmented record is composed of). This is always 1 for fixed-format tables. This value should stay as close to 1.0 as possible. If it gets too big, you can reorganize the table with myisamchk. See section 4.4.6.10 Table Optimization.

Recordblocks

How many blocks (links) are used. For fixed format, this is the same as the number of records.

Deleteblocks

How many blocks (links) are deleted.

Recorddata

How many bytes in the data file are used.

Deleted data

How many bytes in the data file are deleted (unused).

Lost space

If a record is updated to a shorter length, some space is lost. This is the sum of all such losses, in bytes.

Linkdata

When the dynamic table format is used, record fragments are linked with pointers (4 to 7 bytes each). Linkdata is the sum of the amount of storage used by all such pointers.

If a table has been compressed with myisampack, myisamchk -d prints additional information about each table column. See section 4.7.4 myisampack, The MySQL Compressed Read-only Table Generator, for an example of this information and a description of what it means.

4.5 Database Administration Language Reference

4.5.1 OPTIMIZE TABLE Syntax

OPTIMIZE TABLE tbl_name[,tbl_name]...

OPTIMIZE TABLE should be used if you have deleted a large part of a table or if you have made many changes to a table with variable-length rows (tables that have VARCHAR, BLOB, or TEXT columns). Deleted records are maintained in a linked list and subsequent INSERT operations reuse old record positions. You can use OPTIMIZE TABLE to reclaim the unused space and to defragment the data file.

For the moment OPTIMIZE TABLE only works on MyISAM and BDB tables. For BDB tables, OPTIMIZE TABLE is currently mapped to ANALYZE TABLE. See section 4.5.2 ANALYZE TABLE Syntax.

You can get optimize table to work on other table types by starting mysqld with --skip-new or --safe-mode, but in this case OPTIMIZE TABLE is just mapped to ALTER TABLE.

OPTIMIZE TABLE works the following way:

• If the table has deleted or split rows, repair the table.
• If the index pages are not sorted, sort them.
• If the statistics are not up to date (and the repair couldn't be done by sorting the index), update them.

OPTIMIZE TABLE for MyISAM tables is equvialent of running myisamchk --quick --check-changed-tables --sort-index --analyze on the table.

Note that the table is locked during the time OPTIMIZE TABLE is running!

4.5.2 ANALYZE TABLE Syntax

ANALYZE TABLE tbl_name[,tbl_name...]

Analyze and store the key distribution for the table. During the analyze the table is locked with a read lock. This works on MyISAM and BDB tables.

This is equivalent to running myisamchk -a on the table.

MySQL uses the stored key distribution to decide in which order tables should be joined when one does a join on something else than a constant.

The command returns a table with the following columns:

You can check the stored key distribution with the SHOW INDEX command. See section 4.5.5.1 Retrieving information about Database, Tables, Columns, and Indexes.

If the table hasn't changed since the last ANALYZE TABLE command, the table will not be analyzed again.

4.5.3 FLUSH Syntax

FLUSH flush_option [,flush_option]

You should use the FLUSH command if you want to clear some of the internal caches MySQL uses. To execute FLUSH, you must have the RELOAD privilege.

flush_option can be any of the following:

You can also access each of the commands shown above with the mysqladmin utility, using the flush-hosts, flush-logs, reload, or flush-tables commands.

Take also a look at the RESET command used with replication. See section 4.10.6 SQL Commands Related to Replication.

4.5.4 KILL Syntax

KILL thread_id

Each connection to mysqld runs in a separate thread. You can see which threads are running with the SHOW PROCESSLIST command and kill a thread with the KILL thread_id command.

If you have the process privilege, you can see and kill all threads. Otherwise, you can see and kill only your own threads.

You can also use the mysqladmin processlist and mysqladmin kill commands to examine and kill threads.

When you do a KILL, a thread specific kill flag is set for the thread.

In most cases it may take some time for the thread to die as the kill flag is only checked at specific intervals.

• In SELECT, ORDER BY and GROUP BY loops, the flag is checked after reading a block of rows. If the kill flag is set the statement is aborted
• When doing an ALTER TABLE the kill flag is checked before each block of rows are read from the original table. If the kill flag was set the command is aborted and the temporary table is deleted.
• When doing an UPDATE TABLE and DELETE TABLE, the kill flag is checked after each block read and after each updated or delete row. If the kill flag is set the statement is aborted. Note that if you are not using transactions, the changes will not be rolled back!
• GET_LOCK() will abort with NULL.
• An INSERT DELAYED thread will quickly flush all rows it has in memory and die.
• If the thread is in the table lock handler (state: Locked), the table lock will be quickly aborted.
• If the thread is waiting for free disk space in a write call, the write is aborted with an disk full error message.

4.5.5 SHOW Syntax

   SHOW DATABASES [LIKE wild]
or SHOW [OPEN] TABLES [FROM db_name] [LIKE wild]
or SHOW [FULL] COLUMNS FROM tbl_name [FROM db_name] [LIKE wild]
or SHOW INDEX FROM tbl_name [FROM db_name]
or SHOW TABLE STATUS [FROM db_name] [LIKE wild]
or SHOW STATUS [LIKE wild]
or SHOW VARIABLES [LIKE wild]
or SHOW LOGS
or SHOW [FULL] PROCESSLIST
or SHOW GRANTS FOR user
or SHOW CREATE TABLE table_name
or SHOW MASTER STATUS
or SHOW MASTER LOGS
or SHOW SLAVE STATUS

SHOW provides information about databases, tables, columns, or status information about the server. If the LIKE wild part is used, the wild string can be a string that uses the SQL `%' and `_' wild-card characters.

4.5.5.1 Retrieving information about Database, Tables, Columns, and Indexes

You can use db_name.tbl_name as an alternative to the tbl_name FROM db_name syntax. These two statements are equivalent:

mysql> SHOW INDEX FROM mytable FROM mydb;
mysql> SHOW INDEX FROM mydb.mytable;

SHOW DATABASES lists the databases on the MySQL server host. You can also get this list using the mysqlshow command.

SHOW TABLES lists the tables in a given database. You can also get this list using the mysqlshow db_name command.

NOTE: If a user doesn't have any privileges for a table, the table will not show up in the output from SHOW TABLES or mysqlshow db_name.

SHOW OPEN TABLES lists the tables that are currently open in the table cache. See section 5.4.6 How MySQL Opens and Closes Tables. The Comment field tells how many times the table is cached and in_use.

SHOW COLUMNS lists the columns in a given table. If you specify the FULL option, you will also get the privileges you have for each column. If the column types are different than you expect them to be based on a CREATE TABLE statement, note that MySQL sometimes changes column types. See section 6.5.3.1 Silent Column Specification Changes.

The DESCRIBE statement provides information similar to SHOW COLUMNS. See section 6.6.2 DESCRIBE Syntax (Get Information About Columns).

SHOW FIELDS is a synonym for SHOW COLUMNS, and SHOW KEYS is a synonym for SHOW INDEX. You can also list a table's columns or indexes with mysqlshow db_name tbl_name or mysqlshow -k db_name tbl_name.

SHOW INDEX returns the index information in a format that closely resembles the SQLStatistics call in ODBC. The following columns are returned:

Note that as the Cardinality is counted based on statistics stored as integers, it's not necessarily accurate for small tables.

4.5.5.2 SHOW TABLE STATUS

SHOW TABLE STATUS [FROM db_name] [LIKE wild]

SHOW TABLE STATUS (new in Version 3.23) works likes SHOW STATUS, but provides a lot of information about each table. You can also get this list using the mysqlshow --status db_name command. The following columns are returned:

InnoDB tables will report the free space in the tablespace in the table comment.

4.5.5.3 SHOW STATUS

SHOW STATUS provides server status information (like mysqladmin extended-status). The output resembles that shown below, though the format and numbers probably differ:

+--------------------------+------------+
| Variable_name            | Value      |
+--------------------------+------------+
| Aborted_clients          | 0          |
| Aborted_connects         | 0          |
| Bytes_received           | 155372598  |
| Bytes_sent               | 1176560426 |
| Connections              | 30023      |
| Created_tmp_disk_tables  | 0          |
| Created_tmp_tables       | 8340       |
| Created_tmp_files        | 60         |
| Delayed_insert_threads   | 0          |
| Delayed_writes           | 0          |
| Delayed_errors           | 0          |
| Flush_commands           | 1          |
| Handler_delete           | 462604     |
| Handler_read_first       | 105881     |
| Handler_read_key         | 27820558   |
| Handler_read_next        | 390681754  |
| Handler_read_prev        | 6022500    |
| Handler_read_rnd         | 30546748   |
| Handler_read_rnd_next    | 246216530  |
| Handler_update           | 16945404   |
| Handler_write            | 60356676   |
| Key_blocks_used          | 14955      |
| Key_read_requests        | 96854827   |
| Key_reads                | 162040     |
| Key_write_requests       | 7589728    |
| Key_writes               | 3813196    |
| Max_used_connections     | 0          |
| Not_flushed_key_blocks   | 0          |
| Not_flushed_delayed_rows | 0          |
| Open_tables              | 1          |
| Open_files               | 2          |
| Open_streams             | 0          |
| Opened_tables            | 44600      |
| Questions                | 2026873    |
| Select_full_join         | 0          |
| Select_full_range_join   | 0          |
| Select_range             | 99646      |
| Select_range_check       | 0          |
| Select_scan              | 30802      |
| Slave_running            | OFF        |
| Slave_open_temp_tables   | 0          |
| Slow_launch_threads      | 0          |
| Slow_queries             | 0          |
| Sort_merge_passes        | 30         |
| Sort_range               | 500        |
| Sort_rows                | 30296250   |
| Sort_scan                | 4650       |
| Table_locks_immediate    | 1920382    |
| Table_locks_waited       | 0          |
| Threads_cached           | 0          |
| Threads_created          | 30022      |
| Threads_connected        | 1          |
| Threads_running          | 1          |
| Uptime                   | 80380      |
+--------------------------+------------+

The status variables listed above have the following meaning:

Some comments about the above:

• If Opened_tables is big, then your table_cache variable is probably too small.
• If key_reads is big, then your key_cache is probably too small. The cache hit rate can be calculated with key_reads/key_read_requests.
• If Handler_read_rnd is big, then you probably have a lot of queries that require MySQL to scan whole tables or you have joins that don't use keys properly.
• If Threads_created is big, you may want to increase the thread_cache_size variable.

4.5.5.4 SHOW VARIABLES

SHOW VARIABLES [LIKE wild]

SHOW VARIABLES shows the values of some MySQL system variables. You can also get this information using the mysqladmin variables command. If the default values are unsuitable, you can set most of these variables using command-line options when mysqld starts up. See section 4.1.1 mysqld Command-line Options.

The output resembles that shown below, though the format and numbers may differ somewhat:

+-------------------------+---------------------------+
| Variable_name           | Value                     |
+-------------------------+---------------------------+
| ansi_mode               | OFF                       |
| back_log                | 50                        |
| basedir                 | /my/monty/                |
| bdb_cache_size          | 16777216                  |
| bdb_log_buffer_size     | 32768                     |
| bdb_home                | /my/monty/data/           |
| bdb_max_lock            | 10000                     |
| bdb_logdir              |                           |
| bdb_shared_data         | OFF                       |
| bdb_tmpdir              | /tmp/                     |
| binlog_cache_size       | 32768                     |
| concurrent_insert       | ON                        |
| connect_timeout         | 5                         |
| datadir                 | /my/monty/data/           |
| delay_key_write         | ON                        |
| delayed_insert_limit    | 100                       |
| delayed_insert_timeout  | 300                       |
| delayed_queue_size      | 1000                      |
| flush                   | OFF                       |
| flush_time              | 0                         |
| have_bdb                | YES                       |
| have_innodb             | YES                       |
| have_raid               | YES                       |
| have_ssl                | NO                        |
| init_file               |                           |
| interactive_timeout     | 28800                     |
| join_buffer_size        | 131072                    |
| key_buffer_size         | 16776192                  |
| language                | /my/monty/share/english/  |
| large_files_support     | ON                        |
| log                     | OFF                       |
| log_update              | OFF                       |
| log_bin                 | OFF                       |
| log_slave_updates       | OFF                       |
| long_query_time         | 10                        |
| low_priority_updates    | OFF                       |
| lower_case_table_names  | 0                         |
| max_allowed_packet      | 1048576                   |
| max_binlog_cache_size   | 4294967295                |
| max_connections         | 100                       |
| max_connect_errors      | 10                        |
| max_delayed_threads     | 20                        |
| max_heap_table_size     | 16777216                  |
| max_join_size           | 4294967295                |
| max_sort_length         | 1024                      |
| max_tmp_tables          | 32                        |
| max_write_lock_count    | 4294967295                |
| myisam_recover_options  | DEFAULT                   |
| myisam_sort_buffer_size | 8388608                   |
| net_buffer_length       | 16384                     |
| net_read_timeout        | 30                        |
| net_retry_count         | 10                        |
| net_write_timeout       | 60                        |
| open_files_limit        | 0                         |
| pid_file                | /my/monty/data/donna.pid  |
| port                    | 3306                      |
| protocol_version        | 10                        |
| record_buffer           | 131072                    |
| query_buffer_size       | 0                         |
| safe_show_database      | OFF                       |
| server_id               | 0                         |
| skip_locking            | ON                        |
| skip_networking         | OFF                       |
| skip_show_database      | OFF                       |
| slow_launch_time        | 2                         |
| socket                  | /tmp/mysql.sock           |
| sort_buffer             | 2097116                   |
| table_cache             | 64                        |
| table_type              | MYISAM                    |
| thread_cache_size       | 4                         |
| thread_stack            | 65536                     |
| tmp_table_size          | 1048576                   |
| tmpdir                  | /tmp/                     |
| version                 | 3.23.29a-gamma-debug      |
| wait_timeout            | 28800                     |
+-------------------------+---------------------------+

Each option is described below. Values for buffer sizes, lengths, and stack sizes are given in bytes. You can specify values with a suffix of `K' or `M' to indicate kilobytes or megabytes. For example, 16M indicates 16 megabytes. The case of suffix letters does not matter; 16M and 16m are equivalent:

ansi_mode.

Is ON if mysqld was started with --ansi. See section 1.4.3 Running MySQL in ANSI Mode.

back_log

The number of outstanding connection requests MySQL can have. This comes into play when the main MySQL thread gets VERY many connection requests in a very short time. It then takes some time (although very little) for the main thread to check the connection and start a new thread. The back_log value indicates how many requests can be stacked during this short time before MySQL momentarily stops answering new requests. You need to increase this only if you expect a large number of connections in a short period of time. In other words, this value is the size of the listen queue for incoming TCP/IP connections. Your operating system has its own limit on the size of this queue. The manual page for the Unix listen(2) system call should have more details. Check your OS documentation for the maximum value for this variable. Attempting to set back_log higher than your operating system limit will be ineffective.

basedir

The value of the --basedir option.

bdb_cache_size

The buffer that is allocated to cache index and rows for BDB tables. If you don't use BDB tables, you should start mysqld with --skip-bdb to not waste memory for this cache.

bdb_log_buffer_size

The buffer that is allocated to cache index and rows for BDB tables. If you don't use BDB tables, you should set this to 0 or start mysqld with --skip-bdb to not waste memory for this cache.

bdb_home

The value of the --bdb-home option.

bdb_max_lock

The maximum number of locks (1000 by default) you can have active on a BDB table. You should increase this if you get errors of type bdb: Lock table is out of available locks or Got error 12 from ... when you have do long transactions or when mysqld has to examine a lot of rows to calculate the query.

bdb_logdir

The value of the --bdb-logdir option.

bdb_shared_data

Is ON if you are using --bdb-shared-data.

bdb_tmpdir

The value of the --bdb-tmpdir option.

binlog_cache_size. The size of the cache to hold the SQL

statements for the binary log during a transaction. If you often use big, multi-statement transactions you can increase this to get more performance. See section 6.7.1 BEGIN/COMMIT/ROLLBACK Syntax.

character_set

The default character set.

character_sets

The supported character sets.

concurrent_inserts

If ON (the default), MySQL will allow you to use INSERT on MyISAM tables at the same time as you run SELECT queries on them. You can turn this option off by starting mysqld with --safe or --skip-new.

connect_timeout

The number of seconds the mysqld server is waiting for a connect packet before responding with Bad handshake.

datadir

The value of the --datadir option.

delay_key_write

If enabled (is on by default), MySQL will honor the delay_key_write option CREATE TABLE. This means that the key buffer for tables with this option will not get flushed on every index update, but only when a table is closed. This will speed up writes on keys a lot, but you should add automatic checking of all tables with myisamchk --fast --force if you use this. Note that if you start mysqld with the --delay-key-write-for-all-tables option this means that all tables will be treated as if they were created with the delay_key_write option. You can clear this flag by starting mysqld with --skip-new or --safe-mode.

delayed_insert_limit

After inserting delayed_insert_limit rows, the INSERT DELAYED handler will check if there are any SELECT statements pending. If so, it allows these to execute before continuing.

delayed_insert_timeout

How long a INSERT DELAYED thread should wait for INSERT statements before terminating.

delayed_queue_size

What size queue (in rows) should be allocated for handling INSERT DELAYED. If the queue becomes full, any client that does INSERT DELAYED will wait until there is room in the queue again.

flush

This is ON if you have started MySQL with the --flush option.

flush_time

If this is set to a non-zero value, then every flush_time seconds all tables will be closed (to free up resources and sync things to disk). We only recommend this option on Win95, Win98, or on systems where you have very little resources.

have_bdb

YES if mysqld supports Berkeley DB tables. DISABLED if --skip-bdb is used.

have_innodb

YES if mysqld supports InnoDB tables. DISABLED if --skip-innodb is used.

have_raid

YES if mysqld supports the RAID option.

have_ssl

YES if mysqld supports SSL (encryption) on the client/server protocol.

init_file

The name of the file specified with the --init-file option when you start the server. This is a file of SQL statements you want the server to execute when it starts.

interactive_timeout

The number of seconds the server waits for activity on an interactive connection before closing it. An interactive client is defined as a client that uses the CLIENT_INTERACTIVE option to mysql_real_connect(). See also wait_timeout.

join_buffer_size

The size of the buffer that is used for full joins (joins that do not use indexes). The buffer is allocated one time for each full join between two tables. Increase this value to get a faster full join when adding indexes is not possible. (Normally the best way to get fast joins is to add indexes.)

key_buffer_size

Index blocks are buffered and are shared by all threads. key_buffer_size is the size of the buffer used for index blocks. Increase this to get better index handling (for all reads and multiple writes) to as much as you can afford; 64M on a 256M machine that mainly runs MySQL is quite common. If you, however, make this too big (more than 50% of your total memory?) your system may start to page and become REALLY slow. Remember that because MySQL does not cache data read, that you will have to leave some room for the OS filesystem cache. You can check the performance of the key buffer by doing show status and examine the variables Key_read_requests, Key_reads, Key_write_requests, and Key_writes. The Key_reads/Key_read_request ratio should normally be < 0.01. The Key_write/Key_write_requests is usually near 1 if you are using mostly updates/deletes but may be much smaller if you tend to do updates that affect many at the same time or if you are using delay_key_write. See section 4.5.5 SHOW Syntax. To get even more speed when writing many rows at the same time, use LOCK TABLES. See section 6.7.2 LOCK TABLES/UNLOCK TABLES Syntax.

language

The language used for error messages.

large_file_support

If mysqld was compiled with options for big file support.

locked_in_memory

If mysqld was locked in memory with --memlock

log

If logging of all queries is enabled.

log_update

If the update log is enabled.

log_bin

If the binary log is enabled.

log_slave_updates

If the updates from the slave should be logged.

long_query_time

If a query takes longer than this (in seconds), the Slow_queries counter will be incremented. If you are using --log-slow-queries, the query will be logged to the slow query logfile. See section 4.9.5 The Slow Query Log.

lower_case_table_names

If set to 1 table names are stored in lowercase on disk and table names will be case-insensitive. See section 6.1.3 Case Sensitivity in Names.

max_allowed_packet

The maximum size of one packet. The message buffer is initialized to net_buffer_length bytes, but can grow up to max_allowed_packet bytes when needed. This value by default is small, to catch big (possibly wrong) packets. You must increase this value if you are using big BLOB columns. It should be as big as the biggest BLOB you want to use. The current protocol limits max_allowed_packet to 16M.

max_binlog_cache_size

If a multi-statement transaction requires more than this amount of memory, one will get the error "Multi-statement transaction required more than 'max_binlog_cache_size' bytes of storage".

max_binlog_size

Available after 3.23.33. If a write to the binary (replication) log exceeds the given value, rotate the logs. You cannot set it to less than 1024 bytes, or more than 1 GB. Default is 1 GB.

max_connections

The number of simultaneous clients allowed. Increasing this value increases the number of file descriptors that mysqld requires. See below for comments on file descriptor limits. See section A.2.5 Too many connections Error.

max_connect_errors

If there is more than this number of interrupted connections from a host this host will be blocked from further connections. You can unblock a host with the command FLUSH HOSTS.

max_delayed_threads

Don't start more than this number of threads to handle INSERT DELAYED statements. If you try to insert data into a new table after all INSERT DELAYED threads are in use, the row will be inserted as if the DELAYED attribute wasn't specified.

max_heap_table_size

Don't allow creation of heap tables bigger than this.

max_join_size

Joins that are probably going to read more than max_join_size records return an error. Set this value if your users tend to perform joins that lack a WHERE clause, that take a long time, and that return millions of rows.

max_sort_length

The number of bytes to use when sorting BLOB or TEXT values (only the first max_sort_length bytes of each value are used; the rest are ignored).

max_user_connections

The maximum number of active connections for a single user (0 = no limit).

max_tmp_tables

(This option doesn't yet do anything.) Maximum number of temporary tables a client can keep open at the same time.

max_write_lock_count

After this many write locks, allow some read locks to run in between.

myisam_recover_options

The value of the --myisam-recover option.

myisam_sort_buffer_size

The buffer that is allocated when sorting the index when doing a REPAIR or when creating indexes with CREATE INDEX or ALTER TABLE.

myisam_max_extra_sort_file_size.

If the creating of the temporary file for fast index creation would be this much bigger than using the key cache, then prefer the key cache method. This is mainly used to force long character keys in large tables to use the slower key cache method to create the index. NOTE that this parameter is given in megabytes!

myisam_max_sort_file_size

The maximum size of the temporary file MySQL is allowed to use while recreating the index (during REPAIR, ALTER TABLE or LOAD DATA INFILE. If the file size would be bigger than this, the index will be created through the key cache (which is slower). NOTE that this parameter is given in megabytes!

net_buffer_length

The communication buffer is reset to this size between queries. This should not normally be changed, but if you have very little memory, you can set it to the expected size of a query. (That is, the expected length of SQL statements sent by clients. If statements exceed this length, the buffer is automatically enlarged, up to max_allowed_packet bytes.)

net_read_timeout

Number of seconds to wait for more data from a connection before aborting the read. Note that when we don't expect data from a connection, the timeout is defined by write_timeout. See also slave_read_timeout.

net_retry_count

If a read on a communication port is interrupted, retry this many times before giving up. This value should be quite high on FreeBSD as internal interrupts are sent to all threads.

net_write_timeout

Number of seconds to wait for a block to be written to a connection before aborting the write.

open_files_limit

If this is not 0, then mysqld will use this value to reserve file descriptors to use with setrlimit(). If this value is 0 then mysqld will reserve max_connections*5 or max_connections + table_cache*2 (whichever is larger) number of files. You should try increasing this if mysqld gives you the error 'Too many open files'.

pid_file

The value of the --pid-file option.

port

The value of the --port option.

protocol_version

The protocol version used by the MySQL server.

record_buffer

Each thread that does a sequential scan allocates a buffer of this size for each table it scans. If you do many sequential scans, you may want to increase this value.

record_rnd_buffer

When reading rows in sorted order after a sort, the rows are read through this buffer to avoid a disk seeks. If not set, then it's set to the value of record_buffer.

query_buffer_size

The initial allocation of the query buffer. If most of your queries are long (like when inserting blobs), you should increase this!

safe_show_databases

Don't show databases for which the user doesn't have any database or table privileges. This can improve security if you're concerned about people being able to see what databases other users have. See also skip_show_databases.

server_id

The value of the --server-id option.

skip_locking

Is OFF if mysqld uses external locking.

skip_networking

Is ON if we only allow local (socket) connections.

skip_show_databases

This prevents people from doing SHOW DATABASES if they don't have the PROCESS_PRIV privilege. This can improve security if you're concerned about people being able to see what databases other users have. See also safe_show_databases.

slave_read_timeout

Number of seconds to wait for more data from a master/slave connection before aborting the read.

slow_launch_time

If creating the thread takes longer than this value (in seconds), the Slow_launch_threads counter will be incremented.

socket

The Unix socket used by the server.

sort_buffer

Each thread that needs to do a sort allocates a buffer of this size. Increase this value for faster ORDER BY or GROUP BY operations. See section A.4.4 Where MySQL Stores Temporary Files.

table_cache

The number of open tables for all threads. Increasing this value increases the number of file descriptors that mysqld requires. MySQL needs two file descriptors for each unique open table. See below for comments on file descriptor limits. You can check if you need to increase the table cache by checking the Opened_tables variable. See section 4.5.5 SHOW Syntax. If this variable is big and you don't do FLUSH TABLES a lot (which just forces all tables to be closed and reopenend), then you should increase the value of this variable. Make sure that your operating system can handle the number of open file descriptors implied by the table_cache setting. If table_cache is set too high, MySQL may run out of file descriptors and refuse connections, fail to perform queries, and be very unreliable. For information about how the table cache works, see section 5.4.6 How MySQL Opens and Closes Tables.

table_type

The default table type

thread_cache_size

How many threads we should keep in a cache for reuse. When a client disconnects, the client's threads are put in the cache if there aren't more than thread_cache_size threads from before. All new threads are first taken from the cache, and only when the cache is empty is a new thread created. This variable can be increased to improve performance if you have a lot of new connections. (Normally this doesn't give a notable performance improvement if you have a good thread implementation.) By examing the difference between the Connections and Threads_created you can see how efficient the current thread cache is for you.

thread_concurrency

On Solaris, mysqld will call thr_setconcurrency() with this value. thr_setconcurrency() permits the application to give the threads system a hint for the desired number of threads that should be run at the same time.

thread_stack

The stack size for each thread. Many of the limits detected by the crash-me test are dependent on this value. The default is large enough for normal operation. See section 5.1.4 The MySQL Benchmark Suite.

timezone

The timezone for the server.

tmp_table_size

If an in-memory temporary table exceeds this size, MySQL will automatically convert it to an on-disk MyISAM table. Increase the value of tmp_table_size if you do many advanced GROUP BY queries and you have lots of memory.

tmpdir

The directory used for temporary files and temporary tables.

version

The version number for the server.

wait_timeout

The number of seconds the server waits for activity on a connection before closing it. See also interactive_timeout.

The manual section that describes tuning MySQL contains some information of how to tune the above variables. See section 5.5.2 Tuning Server Parameters.

4.5.5.5 SHOW LOGS

SHOW LOGS shows you status information about existing log files. It currently only displays information about Berkeley DB log files.

• File shows the full path to the log file
• Type shows the type of the log file (BDB for Berkeley DB log files)
• Status shows the status of the log file (FREE if the file can be removed, or IN USE if the file is needed by the transaction subsystem)

4.5.5.6 SHOW PROCESSLIST

SHOW PROCESSLIST shows you which threads are running. You can also get this information using the mysqladmin processlist command. If you have the process privilege, you can see all threads. Otherwise, you can see only your own threads. See section 4.5.4 KILL Syntax. If you don't use the FULL option, then only the first 100 characters of each query will be shown.

This command is very useful if you get the 'too many connections' error message and want to find out what's going on. MySQL reserves one extra connection for a client with the Process_priv privilege to ensure that you should always be able to login and check the system (assuming you are not giving this privilege to all your users).

4.5.5.7 SHOW GRANTS

SHOW GRANTS FOR user lists the grant commands that must be issued to duplicate the grants for a user.

mysql> SHOW GRANTS FOR root@localhost;
+---------------------------------------------------------------------+
| Grants for root@localhost                                           |
+---------------------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO 'root'@'localhost' WITH GRANT OPTION |
+---------------------------------------------------------------------+

4.5.5.8 SHOW CREATE TABLE

Shows a CREATE TABLE statement that will create the given table:

mysql> show create table t\G
*************************** 1. row ***************************
       Table: t
Create Table: CREATE TABLE t (
  id int(11) default NULL auto_increment,
  s char(60) default NULL,
  PRIMARY KEY (id)
) TYPE=MyISAM

SHOW CREATE TABLE will quote table and column names according to SQL_QUOTE_SHOW_CREATE option. section 5.5.6 SET Syntax.

4.6 MySQL Localization and International Usage

4.6.1 The Character Set Used for Data and Sorting

By default, MySQL uses the ISO-8859-1 (Latin1) character set with sorting according to Swedish/Finnish. This is the character set suitable in the USA and western Europe.

All standard MySQL binaries are compiled with --with-extra-charsets=complex. This will add code to all standard programs to be able to handle latin1 and all multi-byte character sets within the binary. Other character sets will be loaded from a character-set definition file when needed.

The character set determines what characters are allowed in names and how things are sorted by the ORDER BY and GROUP BY clauses of the SELECT statement.

You can change the character set with the --default-character-set option when you start the server. The character sets available depend on the --with-charset=charset and --with-extra-charset= list-of-charset | complex | all options to configure, and the character set configuration files listed in `SHAREDIR/charsets/Index'. See section 2.3.3 Typical configure Options.

If you change the character set when running MySQL (which may also change the sort order), you must run myisamchk -r -q on all tables. Otherwise your indexes may not be ordered correctly.

When a client connects to a MySQL server, the server sends the default character set in use to the client. The client will switch to use this character set for this connection.

One should use mysql_real_escape_string() when escaping strings for a SQL query. mysql_real_escape_string() is identical to the old mysql_escape_string() function, except that it takes the MYSQL connection handle as the first parameter.

If the client is compiled with different paths than where the server is installed and the user who configured MySQL didn't included all character sets in the MySQL binary, one must specify for the client where it can find the additional character sets it will need if the server runs with a different character set than the client.

One can specify this by putting in a MySQL option file:

[client]
character-sets-dir=/usr/local/mysql/share/mysql/charsets

where the path points to where the dynamic MySQL character sets are stored.

One can force the client to use specific character set by specifying:

[client]
default-character-set=character-set-name

but normally this is never needed.

4.6.1.1 German character set

To get German sorting order, you should start mysqld with --default-character-set=latin_de. This will give you the following characteristics.

When sorting and comparing string's the following mapping is done on the strings before doing the comparison:

ä  ->  ae
ö  ->  oe
ü  ->  ue
ß  ->  ss

All accented characters, are converted to their un-accented uppercase counterpart. All letters are converted to uppercase.

When comparing strings with LIKE the one -> two character mapping is not done. All letters are converted to uppercase. Accent are removed from all letters except: Ü, ü, Ö, ö, Ä and ä.

4.6.2 Non-English Error Messages

mysqld can issue error messages in the following languages: Czech, Danish, Dutch, English (the default), Estonian, French, German, Greek, Hungarian, Italian, Japanese, Korean, Norwegian, Norwegian-ny, Polish, Portuguese, Romanian, Russian, Slovak, Spanish, and Swedish.

To start mysqld with a particular language, use either the --language=lang or -L lang options. For example:

shell> mysqld --language=swedish

or:

shell> mysqld --language=/usr/local/share/swedish

Note that all language names are specified in lowercase.

The language files are located (by default) in `mysql_base_dir/share/LANGUAGE/'.

To update the error message file, you should edit the `errmsg.txt' file and execute the following command to generate the `errmsg.sys' file:

shell> comp_err errmsg.txt errmsg.sys

If you upgrade to a newer version of MySQL, remember to repeat your changes with the new `errmsg.txt' file.

4.6.3 Adding a New Character Set

To add another character set to MySQL, use the following procedure.

Decide if the set is simple or complex. If the character set does not need to use special string collating routines for sorting and does not need multi-byte character support, it is simple. If it needs either of those features, it is complex.

For example, latin1 and danish are simple charactersets while big5 or czech are complex character sets.

In the following section, we have assumed that you name your character set MYSET.

For a simple character set do the following:

1. Add MYSET to the end of the `sql/share/charsets/Index' file Assign an unique number to it.
2. Create the file `sql/share/charsets/MYSET.conf'. (You can use `sql/share/charsets/latin1.conf' as a base for this). The syntax for the file very simple:
   • Comments start with a '#' character and proceed to the end of the line.
   • Words are separated by arbitrary amounts of whitespace.
   • When defining the character set, every word must be a number in hexadecimal format
   • The ctype array takes up the first 257 words. The to_lower, to_upper and sort_order arrays take up 256 words each after that.
   See section 4.6.4 The character definition arrays.
3. Add the character set name to the CHARSETS_AVAILABLE and COMPILED_CHARSETS lists in configure.in.
4. Reconfigure, recompile, and test.

For a complex character set do the following:

1. Create the file `strings/ctype-MYSET.c' in the MySQL source distribution.
2. Add MYSET to the end of the `sql/share/charsets/Index' file. Assign an unique number to it.
3. Look at one of the existing `ctype-*.c' files to see what needs to be defined, for example `strings/ctype-big5.c'. Note that the arrays in your file must have names like ctype_MYSET, to_lower_MYSET, and so on. This corresponds to the arrays in the simple character set. See section 4.6.4 The character definition arrays. For a complex character set
4. Near the top of the file, place a special comment like this:

   /*
    * This comment is parsed by configure to create ctype.c,
    * so don't change it unless you know what you are doing.
    *
    * .configure. number_MYSET=MYNUMBER
    * .configure. strxfrm_multiply_MYSET=N
    * .configure. mbmaxlen_MYSET=N
    */
   

   The configure program uses this comment to include the character set into the MySQL library automatically. The strxfrm_multiply and mbmaxlen lines will be explained in the following sections. Only include them if you the string collating functions or the multi-byte character set functions, respectively.
5. You should then create some of the following functions:
   • my_strncoll_MYSET()
   • my_strcoll_MYSET()
   • my_strxfrm_MYSET()
   • my_like_range_MYSET()
   See section 4.6.5 String Collating Support.
6. Add the character set name to the CHARSETS_AVAILABLE and COMPILED_CHARSETS lists in configure.in.
7. Reconfigure, recompile, and test.

The file `sql/share/charsets/README' includes some more instructions.

If you want to have the character set included in the MySQL distribution, mail a patch to internals@lists.mysql.com.

4.6.4 The character definition arrays

to_lower[] and to_upper[] are simple arrays that hold the lowercase and uppercase characters corresponding to each member of the character set. For example:

to_lower['A'] should contain 'a'
to_upper['a'] should contain 'A'

sort_order[] is a map indicating how characters should be ordered for comparison and sorting purposes. For many character sets, this is the same as to_upper[] (which means sorting will be case insensitive). MySQL will sort characters based on the value of sort_order[character]. For more complicated sorting rules, see the discussion of string collating below. See section 4.6.5 String Collating Support.

ctype[] is an array of bit values, with one element for one character. (Note that to_lower[], to_upper[], and sort_order[] are indexed by character value, but ctype[] is indexed by character value + 1. This is an old legacy to be able to handle EOF.)

You can find the following bitmask definitions in `m_ctype.h':

#define _U      01      /* Uppercase */
#define _L      02      /* Lowercase */
#define _N      04      /* Numeral (digit) */
#define _S      010     /* Spacing character */
#define _P      020     /* Punctuation */
#define _C      040     /* Control character */
#define _B      0100    /* Blank */
#define _X      0200    /* heXadecimal digit */

The ctype[] entry for each character should be the union of the applicable bitmask values that describe the character. For example, 'A' is an uppercase character (_U) as well as a hexadecimal digit (_X), so ctype['A'+1] should contain the value:

_U + _X = 01 + 0200 = 0201

4.6.5 String Collating Support

If the sorting rules for your language are too complex to be handled with the simple sort_order[] table, you need to use the string collating functions.

Right now the best documentation on this is the character sets that are already implemented. Look at the big5, czech, gbk, sjis, and tis160 character sets for examples.

You must specify the strxfrm_multiply_MYSET=N value in the special comment at the top of the file. N should be set to the maximum ratio the strings may grow during my_strxfrm_MYSET (it must be a positive integer).

4.6.6 Multi-byte Character Support

If your want to add support for a new character set that includes multi-byte characters, you need to use the multi-byte character functions.

Right now the best documentation on this is the character sets that are already implemented. Look at the euc_kr, gb2312, gbk, sjis and ujis character sets for examples. These are implemented in the ctype-'charset'.c files in the `strings' directory.

You must specify the mbmaxlen_MYSET=N value in the special comment at the top of the source file. N should be set to the size in bytes of the largest character in the set.

4.7 MySQL Server-Side Scripts and Utilities

4.7.1 Overview of the Server-Side Scripts and Utilities

All MySQL clients that communicate with the server using the mysqlclient library use the following environment variables:

Name	Description
MYSQL_UNIX_PORT	The default socket; used for connections to localhost
MYSQL_TCP_PORT	The default TCP/IP port
MYSQL_PWD	The default password
MYSQL_DEBUG	Debug-trace options when debugging
TMPDIR	The directory where temporary tables/files are created

Use of MYSQL_PWD is insecure. See section 4.2.7 Connecting to the MySQL Server.

The `mysql' client uses the file named in the MYSQL_HISTFILE environment variable to save the command-line history. The default value for the history file is `$HOME/.mysql_history', where $HOME is the value of the HOME environment variable. See section H Environment Variables.

All MySQL programs take many different options. However, every MySQL program provides a --help option that you can use to get a full description of the program's different options. For example, try mysql --help.

You can override default options for all standard client programs with an option file. section 4.1.2 my.cnf Option Files.

The list below briefly describes the MySQL programs:

myisamchk

Utility to describe, check, optimize, and repair MySQL tables. Because myisamchk has many functions, it is described in its own chapter. See section 4 MySQL Database Administration.

make_binary_distribution

Makes a binary release of a compiled MySQL. This could be sent by FTP to `/pub/mysql/Incoming' on support.mysql.com for the convenience of other MySQL users.

msql2mysql

A shell script that converts mSQL programs to MySQL. It doesn't handle all cases, but it gives a good start when converting.

mysqlaccess

A script that checks the access privileges for a host, user, and database combination.

mysqladmin

Utility for performing administrative operations, such as creating or dropping databases, reloading the grant tables, flushing tables to disk, and reopening log files. mysqladmin can also be used to retrieve version, process, and status information from the server. See section 4.8.3 mysqladmin, Administrating a MySQL Server.

mysqlbug

The MySQL bug report script. This script should always be used when filing a bug report to the MySQL list.

mysqld

The SQL daemon. This should always be running.

mysqldump

Dumps a MySQL database into a file as SQL statements or as tab-separated text files. Enhanced freeware originally by Igor Romanenko. See section 4.8.5 mysqldump, Dumping Table Structure and Data.

mysqlimport

Imports text files into their respective tables using LOAD DATA INFILE. See section 4.8.7 mysqlimport, Importing Data from Text Files.

mysqlshow

Displays information about databases, tables, columns, and indexes.

mysql_install_db

Creates the MySQL grant tables with default privileges. This is usually executed only once, when first installing MySQL on a system.

replace

A utility program that is used by msql2mysql, but that has more general applicability as well. replace changes strings in place in files or on the standard input. Uses a finite state machine to match longer strings first. Can be used to swap strings. For example, this command swaps a and b in the given files:

shell> replace a b b a -- file1 file2 ...

4.7.2 safe_mysqld, the wrapper around mysqld

safe_mysqld is the recommended way to start a mysqld daemon on Unix. safe_mysqld adds some safety features such as restarting the server when an error occurs and logging run-time information to a log file.

If you don't use --mysqld=# or --mysqld-version=# safe_mysqld will use an executable named mysqld-max if it exists. If not, safe_mysqld will start mysqld. This makes it very easy to test to use mysqld-max instead of mysqld; Just copy mysqld-max to where you have mysqld and it will be used.

Normally one should never edit the safe_mysqld script, but instead put the options to safe_mysqld in the [safe_mysqld] section in the my.cnf file. safe_mysqld will read all options from the [mysqld], [server] and [safe_mysqld] sections from the option files. See section 4.1.2 my.cnf Option Files.

Note that all options on the command line to safe_mysqld are passed to mysqld. If you wants to use any options in safe_mysqld that mysqld doesn't support, you must specify these in the option file.

Most of the options to safe_mysqld are the same as the options to mysqld. See section 4.1.1 mysqld Command-line Options.

safe_mysqld supports the following options:

--basedir=path

--core-file-size=#

Size of the core file mysqld should be able to create. Passed to ulimit -c.

--datadir=path

--defaults-extra-file=path

--defaults-file=path

--err-log=path

--ledir=path

Path to mysqld

--log=path

--mysqld=mysqld-version

Name of the mysqld version in the ledir directory you want to start.

--mysqld-version=version

Similar to --mysqld= but here you only give the suffix for mysqld. For example if you use --mysqld-version=max, safe_mysqld will start the ledir/mysqld-max version. If the argument to --mysqld-version is empty, ledir/mysqld will be used.

--no-defaults

--open-files-limit=#

Number of files mysqld should be able to open. Passed to ulimit -n. Note that you need to start safe_mysqld as root for this to work properly!

--pid-file=path

--port=#

--socket=path

--timezone=#

Set the timezone (the TZ) variable to the value of this parameter.

--user=#

The safe_mysqld script is written so that it normally is able to start a server that was installed from either a source or a binary version of MySQL, even if these install the server in slightly different locations. safe_mysqld expects one of these conditions to be true:

• The server and databases can be found relative to the directory from which safe_mysqld is invoked. safe_mysqld looks under its working directory for `bin' and `data' directories (for binary distributions) or for `libexec' and `var' directories (for source distributions). This condition should be met if you execute safe_mysqld from your MySQL installation directory (for example, `/usr/local/mysql' for a binary distribution).
• If the server and databases cannot be found relative to the working directory, safe_mysqld attempts to locate them by absolute pathnames. Typical locations are `/usr/local/libexec' and `/usr/local/var'. The actual locations are determined when the distribution was built from which safe_mysqld comes. They should be correct if MySQL was installed in a standard location.

Because safe_mysqld will try to find the server and databases relative to its own working directory, you can install a binary distribution of MySQL anywhere, as long as you start safe_mysqld from the MySQL installation directory:

shell> cd mysql_installation_directory
shell> bin/safe_mysqld &

If safe_mysqld fails, even when invoked from the MySQL installation directory, you can modify it to use the path to mysqld and the pathname options that are correct for your system. Note that if you upgrade MySQL in the future, your modified version of safe_mysqld will be overwritten, so you should make a copy of your edited version that you can reinstall.

4.7.3 mysqld_multi, program for managing multiple MySQL servers

mysqld_multi is meant for managing several mysqld processes running in different UNIX sockets and TCP/IP ports.

The program will search for group(s) named [mysqld#] from my.cnf (or the given --config-file=...), where # can be any positive number starting from 1. These groups should be the same as the usual [mysqld] group (e.g. options to mysqld, see MySQL manual for detailed information about this group), but with those port, socket etc. options that are wanted for each separate mysqld processes. The number in the group name has another function; it can be used for starting, stopping, or reporting some specific mysqld servers with this program. See the usage and options below for more information.

Usage: mysqld_multi [OPTIONS] {start|stop|report} [GNR,GNR,GNR...]
or     mysqld_multi [OPTIONS] {start|stop|report} [GNR-GNR,GNR,GNR-GNR,...]

The GNR above means the group number. You can start, stop or report any GNR, or several of them at the same time. (See --example) The GNRs list can be comma separated, or a dash combined, of which the latter means that all the GNRs between GNR1-GNR2 will be affected. Without GNR argument all the found groups will be either started, stopped, or reported. Note that you must not have any white spaces in the GNR list. Anything after a white space is ignored.

mysqld_multi supports the following options:

--config-file=...

Alternative config file. NOTE: This will not affect this program's own options (group [mysqld_multi]), but only groups [mysqld#]. Without this option everything will be searched from the ordinary my.cnf file.

--example

Give an example of a config file.

--help

Print this help and exit.

--log=...

Log file. Full path to and the name for the log file. NOTE: If the file exists, everything will be appended.

--mysqladmin=...

mysqladmin binary to be used for a server shutdown.

--mysqld=...

mysqld binary to be used. Note that you can give safe_mysqld to this option also. The options are passed to mysqld. Just make sure you have mysqld in your environment variable PATH or fix safe_mysqld.

--no-log

Print to stdout instead of the log file. By default the log file is turned on.

--password=...

Password for user for mysqladmin.

--tcp-ip

Connect to the MySQL server(s) via the TCP/IP port instead of the UNIX socket. This affects stopping and reporting. If a socket file is missing, the server may still be running, but can be accessed only via the TCP/IP port. By default connecting is done via the UNIX socket.

--user=...

MySQL user for mysqladmin.

--version

Print the version number and exit.

Some notes about mysqld_multi:

• Make sure that the MySQL user, who is stopping the mysqld services (e.g using the mysqladmin) have the same password and username for all the data directories accessed (to the 'mysql' database) And make sure that the user has the 'Shutdown_priv' privilege! If you have many data- directories and many different 'mysql' databases with different passwords for the MySQL 'root' user, you may want to create a common 'multi_admin' user for each using the same password (see below). Example how to do it:

  shell> mysql -u root -S /tmp/mysql.sock -proot_password -e
  "GRANT SHUTDOWN ON *.* TO multi_admin@localhost IDENTIFIED BY 'multipass'"
  See section 4.2.5  How the Privilege System Works.
  

  You will have to do the above for each mysqld running in each data directory, that you have (just change the socket, -S=...)
• pid-file is very important, if you are using safe_mysqld to start mysqld (e.g. --mysqld=safe_mysqld) Every mysqld should have its own pid-file. The advantage using safe_mysqld instead of mysqld directly here is, that safe_mysqld 'guards' every mysqld process and will restart it, if a mysqld process fails due to signal kill -9, or similar. (Like segmentation fault, which MySQL should never do, of course ;) Please note that safe_mysqld script may require that you start it from a certain place. This means that you may have to CD to a certain directory, before you start the mysqld_multi. If you have problems starting, please see the safe_mysqld script. Check especially the lines:

  --------------------------------------------------------------------------
  MY_PWD=`pwd` Check if we are starting this relative (for the binary
  release) if test -d /data/mysql -a -f ./share/mysql/english/errmsg.sys
  -a -x ./bin/mysqld
  --------------------------------------------------------------------------
  See section 4.7.2  safe_mysqld, the wrapper around mysqld.
  

  The above test should be successful, or you may encounter problems.
• Beware of the dangers starting multiple mysqlds in the same data directory. Use separate data directories, unless you KNOW what you are doing!
• The socket file and the TCP/IP port must be different for every mysqld.
• The first and fifth mysqld group were intentionally left out from the example. You may have 'gaps' in the config file. This gives you more flexibility. The order in which the mysqlds are started or stopped depends on the order in which they appear in the config file.
• When you want to refer to a certain group using GNR with this program, just use the number in the end of the group name ( [mysqld# <== ).
• You may want to use option '--user' for mysqld, but in order to do this you need to be root when you start the mysqld_multi script. Having the option in the config file doesn't matter; you will just get a warning, if you are not the superuser and the mysqlds are started under YOUR UNIX account. IMPORTANT: Make sure that the pid-file and the data directory are read+write(+execute for the latter one) accessible for THAT UNIX user, who the specific mysqld process is started as. DON'T use the UNIX root account for this, unless you KNOW what you are doing!
• MOST IMPORTANT: Make sure that you understand the meanings of the options that are passed to the mysqlds and why WOULD YOU WANT to have separate mysqld processes. Starting multiple mysqlds in one data directory WILL NOT give you extra performance in a threaded system!

See section 4.1.4 Running Multiple MySQL Servers on the Same Machine.

This is an example of the config file on behalf of mysqld_multi.

# This file should probably be in your home dir (~/.my.cnf) or /etc/my.cnf
# Version 2.1 by Jani Tolonen

[mysqld_multi]
mysqld     = /usr/local/bin/safe_mysqld
mysqladmin = /usr/local/bin/mysqladmin
user       = multi_admin
password   = multipass

[mysqld2]
socket     = /tmp/mysql.sock2
port       = 3307
pid-file   = /usr/local/mysql/var2/hostname.pid2
datadir    = /usr/local/mysql/var2
language   = /usr/local/share/mysql/english
user       = john

[mysqld3]
socket     = /tmp/mysql.sock3
port       = 3308
pid-file   = /usr/local/mysql/var3/hostname.pid3
datadir    = /usr/local/mysql/var3
language   = /usr/local/share/mysql/swedish
user       = monty

[mysqld4]
socket     = /tmp/mysql.sock4
port       = 3309
pid-file   = /usr/local/mysql/var4/hostname.pid4
datadir    = /usr/local/mysql/var4
language   = /usr/local/share/mysql/estonia
user       = tonu

[mysqld6]
socket     = /tmp/mysql.sock6
port       = 3311
pid-file   = /usr/local/mysql/var6/hostname.pid6
datadir    = /usr/local/mysql/var6
language   = /usr/local/share/mysql/japanese
user       = jani

See section 4.1.2 my.cnf Option Files.

4.7.4 myisampack, The MySQL Compressed Read-only Table Generator

myisampack is used to compress MyISAM tables, and pack_isam is used to compress ISAM tables. Because ISAM tables are deprecated, we will only discuss myisampack here, but everything said about myisampack should also be true for pack_isam.

myisampack works by compressing each column in the table separately. The information needed to decompress columns is read into memory when the table is opened. This results in much better performance when accessing individual records, because you only have to uncompress exactly one record, not a much larger disk block as when using Stacker on MS-DOS. Usually, myisampack packs the data file 40%-70%.

MySQL uses memory mapping (mmap()) on compressed tables and falls back to normal read/write file usage if mmap() doesn't work.

There are currently two limitations with myisampack:

• After packing, the table is read-only.
• myisampack can also pack BLOB or TEXT columns. The older pack_isam could not do this.

Fixing these limitations is on our TODO list but with low priority.

myisampack is invoked like this:

shell> myisampack [options] filename ...

Each filename should be the name of an index (`.MYI') file. If you are not in the database directory, you should specify the pathname to the file. It is permissible to omit the `.MYI' extension.

myisampack supports the following options:

-b, --backup

Make a backup of the table as tbl_name.OLD.

-#, --debug=debug_options

Output debug log. The debug_options string often is 'd:t:o,filename'.

-f, --force

Force packing of the table even if it becomes bigger or if the temporary file exists. myisampack creates a temporary file named `tbl_name.TMD' while it compresses the table. If you kill myisampack, the `.TMD' file may not be deleted. Normally, myisampack exits with an error if it finds that `tbl_name.TMD' exists. With --force, myisampack packs the table anyway.

-?, --help

Display a help message and exit.

-j big_tbl_name, --join=big_tbl_name

Join all tables named on the command line into a single table big_tbl_name. All tables that are to be combined MUST be identical (same column names and types, same indexes, etc.).

-p #, --packlength=#

Specify the record length storage size, in bytes. The value should be 1, 2, or 3. (myisampack stores all rows with length pointers of 1, 2, or 3 bytes. In most normal cases, myisampack can determine the right length value before it begins packing the file, but it may notice during the packing process that it could have used a shorter length. In this case, myisampack will print a note that the next time you pack the same file, you could use a shorter record length.)

-s, --silent

Silent mode. Write output only when errors occur.

-t, --test

Don't actually pack table, just test packing it.

-T dir_name, --tmp_dir=dir_name

Use the named directory as the location in which to write the temporary table.

-v, --verbose

Verbose mode. Write information about progress and packing result.

-V, --version

Display version information and exit.

-w, --wait

Wait and retry if table is in use. If the mysqld server was invoked with the --skip-locking option, it is not a good idea to invoke myisampack if the table might be updated during the packing process.

The sequence of commands shown below illustrates a typical table compression session:

shell> ls -l station.*
-rw-rw-r--   1 monty    my         994128 Apr 17 19:00 station.MYD
-rw-rw-r--   1 monty    my          53248 Apr 17 19:00 station.MYI
-rw-rw-r--   1 monty    my           5767 Apr 17 19:00 station.frm

shell> myisamchk -dvv station

MyISAM file:     station
Isam-version:  2
Creation time: 1996-03-13 10:08:58
Recover time:  1997-02-02  3:06:43
Data records:              1192  Deleted blocks:              0
Datafile: Parts:           1192  Deleted data:                0
Datafile pointer (bytes):     2  Keyfile pointer (bytes):     2
Max datafile length:   54657023  Max keyfile length:   33554431
Recordlength:               834
Record format: Fixed length

table description:
Key Start Len Index   Type                       Root  Blocksize    Rec/key
1   2     4   unique  unsigned long              1024       1024          1
2   32    30  multip. text                      10240       1024          1

Field Start Length Type
1     1     1
2     2     4
3     6     4
4     10    1
5     11    20
6     31    1
7     32    30
8     62    35
9     97    35
10    132   35
11    167   4
12    171   16
13    187   35
14    222   4
15    226   16
16    242   20
17    262   20
18    282   20
19    302   30
20    332   4
21    336   4
22    340   1
23    341   8
24    349   8
25    357   8
26    365   2
27    367   2
28    369   4
29    373   4
30    377   1
31    378   2
32    380   8
33    388   4
34    392   4
35    396   4
36    400   4
37    404   1
38    405   4
39    409   4
40    413   4
41    417   4
42    421   4
43    425   4
44    429   20
45    449   30
46    479   1
47    480   1
48    481   79
49    560   79
50    639   79
51    718   79
52    797   8
53    805   1
54    806   1
55    807   20
56    827   4
57    831   4

shell> myisampack station.MYI
Compressing station.MYI: (1192 records)
- Calculating statistics

normal:     20  empty-space:      16  empty-zero:        12  empty-fill:  11
pre-space:   0  end-space:        12  table-lookups:      5  zero:         7
Original trees:  57  After join: 17
- Compressing file
87.14%

shell> ls -l station.*
-rw-rw-r--   1 monty    my         127874 Apr 17 19:00 station.MYD
-rw-rw-r--   1 monty    my          55296 Apr 17 19:04 station.MYI
-rw-rw-r--   1 monty    my           5767 Apr 17 19:00 station.frm

shell> myisamchk -dvv station

MyISAM file:     station
Isam-version:  2
Creation time: 1996-03-13 10:08:58
Recover time:  1997-04-17 19:04:26
Data records:              1192  Deleted blocks:              0
Datafile: Parts:           1192  Deleted data:                0
Datafilepointer (bytes):      3  Keyfile pointer (bytes):     1
Max datafile length:   16777215  Max keyfile length:     131071
Recordlength:               834
Record format: Compressed

table description:
Key Start Len Index   Type                       Root  Blocksize    Rec/key
1   2     4   unique  unsigned long             10240       1024          1
2   32    30  multip. text                      54272       1024          1

Field Start Length Type                         Huff tree  Bits
1     1     1      constant                             1     0
2     2     4      zerofill(1)                          2     9
3     6     4      no zeros, zerofill(1)                2     9
4     10    1                                           3     9
5     11    20     table-lookup                         4     0
6     31    1                                           3     9
7     32    30     no endspace, not_always              5     9
8     62    35     no endspace, not_always, no empty    6     9
9     97    35     no empty                             7     9
10    132   35     no endspace, not_always, no empty    6     9
11    167   4      zerofill(1)                          2     9
12    171   16     no endspace, not_always, no empty    5     9
13    187   35     no endspace, not_always, no empty    6     9
14    222   4      zerofill(1)                          2     9
15    226   16     no endspace, not_always, no empty    5     9
16    242   20     no endspace, not_always              8     9
17    262   20     no endspace, no empty                8     9
18    282   20     no endspace, no empty                5     9
19    302   30     no endspace, no empty                6     9
20    332   4      always zero                          2     9
21    336   4      always zero                          2     9
22    340   1                                           3     9
23    341   8      table-lookup                         9     0
24    349   8      table-lookup                        10     0
25    357   8      always zero                          2     9
26    365   2                                           2     9
27    367   2      no zeros, zerofill(1)                2     9
28    369   4      no zeros, zerofill(1)                2     9
29    373   4      table-lookup                        11     0
30    377   1                                           3     9
31    378   2      no zeros, zerofill(1)                2     9
32    380   8      no zeros                             2     9
33    388   4      always zero                          2     9
34    392   4      table-lookup                        12     0
35    396   4      no zeros, zerofill(1)               13     9
36    400   4      no zeros, zerofill(1)                2     9
37    404   1                                           2     9
38    405   4      no zeros                             2     9
39    409   4      always zero                          2     9
40    413   4      no zeros                             2     9
41    417   4      always zero                          2     9
42    421   4      no zeros                             2     9
43    425   4      always zero                          2     9
44    429   20     no empty                             3     9
45    449   30     no empty                             3     9
46    479   1                                          14     4
47    480   1                                          14     4
48    481   79     no endspace, no empty               15     9
49    560   79     no empty                             2     9
50    639   79     no empty                             2     9
51    718   79     no endspace                         16     9
52    797   8      no empty                             2     9
53    805   1                                          17     1
54    806   1                                           3     9
55    807   20     no empty                             3     9
56    827   4      no zeros, zerofill(2)                2     9
57    831   4      no zeros, zerofill(1)                2     9

The information printed by myisampack is described below:

normal

The number of columns for which no extra packing is used.

empty-space

The number of columns containing values that are only spaces; these will occupy 1 bit.

empty-zero

The number of columns containing values that are only binary 0's; these will occupy 1 bit.

empty-fill

The number of integer columns that don't occupy the full byte range of their type; these are changed to a smaller type (for example, an INTEGER column may be changed to MEDIUMINT).

pre-space

The number of decimal columns that are stored with leading spaces. In this case, each value will contain a count for the number of leading spaces.

end-space

The number of columns that have a lot of trailing spaces. In this case, each value will contain a count for the number of trailing spaces.

table-lookup

The column had only a small number of different values, which were converted to an ENUM before Huffman compression.

zero

The number of columns for which all values are zero.

Original trees

The initial number of Huffman trees.

After join

The number of distinct Huffman trees left after joining trees to save some header space.

After a table has been compressed, myisamchk -dvv prints additional information about each field:

Type

The field type may contain the following descriptors:

constant

All rows have the same value.

no endspace

Don't store endspace.

no endspace, not_always

Don't store endspace and don't do end space compression for all values.

no endspace, no empty

Don't store endspace. Don't store empty values.

table-lookup

The column was converted to an ENUM.

zerofill(n)

The most significant n bytes in the value are always 0 and are not stored.

no zeros

Don't store zeros.

always zero

0 values are stored in 1 bit.

Huff tree

The Huffman tree associated with the field.

Bits

The number of bits used in the Huffman tree.

After you have run pack_isam/myisampack you must run isamchk/myisamchk to re-create the index. At this time you can also sort the index blocks and create statistics needed for the MySQL optimizer to work more efficiently:

myisamchk -rq --analyze --sort-index table_name.MYI
isamchk   -rq --analyze --sort-index table_name.ISM

After you have installed the packed table into the MySQL database directory you should do mysqladmin flush-tables to force mysqld to start using the new table.

If you want to unpack a packed table, you can do this with the --unpack option to isamchk or myisamchk.

4.7.5 mysqld-max, An extended mysqld server

mysqld-max is the MySQL server (mysqld) configured with the following configure options:

You can find the MySQL-max binaries at http://www.mysql.com/downloads/mysql-max-3.23.html.

The Windows MySQL 3.23 binary distribution includes both the standard mysqld.exe binary and the mysqld-max.exe binary. http://www.mysql.com/downloads/mysql-3.23.html. See section 2.1.2 Installing MySQL on Windows.

Note that as Berkeley DB and InnoDB are not available for all platforms, some of the Max binaries may not have support for both of these. You can check which table types are supported by doing the following query:

mysql> show variables like "have_%";
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| have_bdb      | YES   |
| have_innodb   | NO    |
| have_isam     | YES   |
| have_raid     | NO    |
| have_ssl      | NO    |
+---------------+-------+

The meaning of the values are:

NOTE: To be able to create InnoDB tables you MUST edit your startup options to include at least the innodb_data_file_path option. See section 7.6.2 InnoDB startup options.

To get better performance for BDB tables, you should add some configuration options for these too. See section 7.5.3 BDB startup options.

safe_mysqld will automatically try to start any mysqld binary with the -max prefix. This makes it very easy to test out a another mysqld binary in an existing installation. Just run configure with the options you want and then install the new mysqld binary as mysqld-max in the same directory where your old mysqld binary is. See section 4.7.2 safe_mysqld, the wrapper around mysqld.

The mysqld-max RPM uses the above mentioned safe_mysqld feature. It just installs the mysqld-max executable and safe_mysqld will automatically use this executable when safe_mysqld is restarted.

The following table shows which table types our standard MySQL-Max binaries includes:

4.8 MySQL Client-Side Scripts and Utilities

4.8.1 Overview of the Client-Side Scripts and Utilities

All MySQL clients that communicate with the server using the mysqlclient library use the following environment variables:

Name	Description
MYSQL_UNIX_PORT	The default socket; used for connections to localhost
MYSQL_TCP_PORT	The default TCP/IP port
MYSQL_PWD	The default password
MYSQL_DEBUG	Debug-trace options when debugging
TMPDIR	The directory where temporary tables/files are created

Use of MYSQL_PWD is insecure. See section 4.2.7 Connecting to the MySQL Server.

The `mysql' client uses the file named in the MYSQL_HISTFILE environment variable to save the command-line history. The default value for the history file is `$HOME/.mysql_history', where $HOME is the value of the HOME environment variable. See section H Environment Variables.

All MySQL programs take many different options. However, every MySQL program provides a --help option that you can use to get a full description of the program's different options. For example, try mysql --help.

You can override default options for all standard client programs with an option file. section 4.1.2 my.cnf Option Files.

The list below briefly describes the MySQL programs:

myisamchk

Utility to describe, check, optimize, and repair MySQL tables. Because myisamchk has many functions, it is described in its own chapter. See section 4 MySQL Database Administration.

make_binary_distribution

Makes a binary release of a compiled MySQL. This could be sent by FTP to `/pub/mysql/Incoming' on support.mysql.com for the convenience of other MySQL users.

msql2mysql

A shell script that converts mSQL programs to MySQL. It doesn't handle all cases, but it gives a good start when converting.

mysqlaccess

A script that checks the access privileges for a host, user, and database combination.

mysqladmin

Utility for performing administrative operations, such as creating or dropping databases, reloading the grant tables, flushing tables to disk, and reopening log files. mysqladmin can also be used to retrieve version, process, and status information from the server. See section 4.8.3 mysqladmin, Administrating a MySQL Server.

mysqlbug

The MySQL bug report script. This script should always be used when filing a bug report to the MySQL list.

mysqld

The SQL daemon. This should always be running.

mysqldump

Dumps a MySQL database into a file as SQL statements or as tab-separated text files. Enhanced freeware originally by Igor Romanenko. See section 4.8.5 mysqldump, Dumping Table Structure and Data.

mysqlimport

Imports text files into their respective tables using LOAD DATA INFILE. See section 4.8.7 mysqlimport, Importing Data from Text Files.

mysqlshow

Displays information about databases, tables, columns, and indexes.

mysql_install_db

Creates the MySQL grant tables with default privileges. This is usually executed only once, when first installing MySQL on a system.

replace

A utility program that is used by msql2mysql, but that has more general applicability as well. replace changes strings in place in files or on the standard input. Uses a finite state machine to match longer strings first. Can be used to swap strings. For example, this command swaps a and b in the given files:

shell> replace a b b a -- file1 file2 ...

4.8.2 The Command-line Tool

mysql is a simple SQL shell (with GNU readline capabilities). It supports interactive and non-interactive use. When used interactively, query results are presented in an ASCII-table format. When used non-interactively (for example, as a filter), the result is presented in tab-separated format. (The output format can be changed using command-line options.) You can run scripts simply like this:

shell> mysql database < script.sql > output.tab

If you have problems due to insufficient memory in the client, use the --quick option! This forces mysql to use mysql_use_result() rather than mysql_store_result() to retrieve the result set.

Using mysql is very easy. Just start it as follows: mysql database or mysql --user=user_name --password=your_password database. Type a SQL statement, end it with `;', `\g', or `\G' and press RETURN/ENTER.

mysql supports the following options:

-?, --help

Display this help and exit.

-A, --no-auto-rehash

No automatic rehashing. One has to use 'rehash' to get table and field completion. This gives a quicker start of mysql.

-B, --batch

Print results with a tab as separator, each row on a new line. Doesn't use history file.

--character-sets-dir=...

Directory where character sets are located.

-C, --compress

Use compression in server/client protocol.

-#, --debug[=...]

Debug log. Default is 'd:t:o,/tmp/mysql.trace'.

-D, --database=...

Database to use. This is mainly useful in the my.cnf file.

--default-character-set=...

Set the default character set.

-e, --execute=...

Execute command and quit. (Output like with --batch)

-E, --vertical

Print the output of a query (rows) vertically. Without this option you can also force this output by ending your statements with \G.

-f, --force

Continue even if we get a SQL error.

-g, --no-named-commands

Named commands are disabled. Use \* form only, or use named commands only in the beginning of a line ending with a semicolon (;). Since Version 10.9, the client now starts with this option ENABLED by default! With the -g option, long format commands will still work from the first line, however.

-G, --enable-named-commands

Named commands are enabled. Long format commands are allowed as well as shortened \* commands.

-i, --ignore-space

Ignore space after function names.

-h, --host=...

Connect to the given host.

-H, --html

Produce HTML output.

-L, --skip-line-numbers

Don't write line number for errors. Useful when one wants to compare result files that includes error messages

--no-pager

Disable pager and print to stdout. See interactive help (\h) also.

--no-tee

Disable outfile. See interactive help (\h) also.

-n, --unbuffered

Flush buffer after each query.

-N, --skip-column-names

Don't write column names in results.

-O, --set-variable var=option

Give a variable a value. --help lists variables.

-o, --one-database

Only update the default database. This is useful for skipping updates to other database in the update log.

--pager[=...]

Output type. Default is your ENV variable PAGER. Valid pagers are less, more, cat [> filename], etc. See interactive help (\h) also. This option does not work in batch mode. Pager works only in UNIX.

-p[password], --password[=...]

Password to use when connecting to server. If a password is not given on the command line, you will be prompted for it. Note that if you use the short form -p you can't have a space between the option and the password.

-P --port=...

TCP/IP port number to use for connection.

-q, --quick

Don't cache result, print it row-by-row. This may slow down the server if the output is suspended. Doesn't use history file.

-r, --raw

Write column values without escape conversion. Used with --batch

-s, --silent

Be more silent.

-S --socket=...

Socket file to use for connection.

-t --table

Output in table format. This is default in non-batch mode.

-T, --debug-info

Print some debug information at exit.

--tee=...

Append everything into outfile. See interactive help (\h) also. Does not work in batch mode.

-u, --user=#

User for login if not current user.

-U, --safe-updates[=#], --i-am-a-dummy[=#]

Only allow UPDATE and DELETE that uses keys. See below for more information about this option. You can reset this option if you have it in your my.cnf file by using --safe-updates=0.

-v, --verbose

More verbose output (-v -v -v gives the table output format).

-V, --version

Output version information and exit.

-w, --wait

Wait and retry if connection is down instead of aborting.

You can also set the following variables with -O or --set-variable:

Variable Name	Default	Description
connect_timeout	0	Number of seconds before timeout connection.
max_allowed_packet	16777216	Max packetlength to send/receive from to server
net_buffer_length	16384	Buffer for TCP/IP and socket communication
select_limit	1000	Automatic limit for SELECT when using --i-am-a-dummy
max_join_size	1000000	Automatic limit for rows in a join when using --i-am-a-dummy.

If you type 'help' on the command line, mysql will print out the commands that it supports:

mysql> help

MySQL commands:
help    (\h)    Display this text.
?       (\h)    Synonym for `help'.
clear   (\c)    Clear command.
connect (\r)    Reconnect to the server. Optional arguments are db and host.
edit    (\e)    Edit command with $EDITOR.
ego     (\G)    Send command to mysql server, display result vertically.
exit    (\q)    Exit mysql. Same as quit.
go      (\g)    Send command to mysql server.
nopager (\n)    Disable pager, print to stdout.
notee   (\t)    Don't write into outfile.
pager   (\P)    Set PAGER [to_pager]. Print the query results via PAGER.
print   (\p)    Print current command.
quit    (\q)    Quit mysql.
rehash  (\#)    Rebuild completion hash.
source  (\.)    Execute a SQL script file. Takes a file name as an argument.
status  (\s)    Get status information from the server.
tee     (\T)    Set outfile [to_outfile]. Append everything into given outfile.
use     (\u)    Use another database. Takes database name as argument.

From the above, pager only works in UNIX.

The status command gives you some information about the connection and the server you are using. If you are running in the --safe-updates mode, status will also print the values for the mysql variables that affect your queries.

A useful startup option for beginners (introduced in MySQL Version 3.23.11) is --safe-updates (or --i-am-a-dummy for users that has at some time done a DELETE FROM table_name but forgot the WHERE clause). When using this option, mysql sends the following command to the MySQL server when opening the connection:

SET SQL_SAFE_UPDATES=1,SQL_SELECT_LIMIT=#select_limit#,
    SQL_MAX_JOIN_SIZE=#max_join_size#"

where #select_limit# and #max_join_size# are variables that can be set from the mysql command line. See section 5.5.6 SET Syntax.

The effect of the above is:

• You are not allowed to do an UPDATE or DELETE statement if you don't have a key constraint in the WHERE part. One can, however, force an UPDATE/DELETE by using LIMIT:

  UPDATE table_name SET not_key_column=# WHERE not_key_column=# LIMIT 1;
  

• All big results are automatically limited to #select_limit# rows.
• SELECT's that will probably need to examine more than #max_join_size row combinations will be aborted.

Some useful hints about the mysql client:

Some data is much more readable when displayed vertically, instead of the usual horizontal box type output. For example longer text, which includes new lines, is often much easier to be read with vertical output.

mysql> select * from mails where length(txt) < 300 limit 300,1\G
*************************** 1. row ***************************
  msg_nro: 3068
     date: 2000-03-01 23:29:50
time_zone: +0200
mail_from: Monty
    reply: monty@no.spam.com
  mail_to: "Thimble Smith" <tim@no.spam.com>
      sbj: UTF-8
      txt: >>>>> "Thimble" == Thimble Smith writes:

Thimble> Hi.  I think this is a good idea.  Is anyone familiar with UTF-8
Thimble> or Unicode?  Otherwise I'll put this on my TODO list and see what
Thimble> happens.

Yes, please do that.

Regards,
Monty
     file: inbox-jani-1
     hash: 190402944
1 row in set (0.09 sec)

• For logging, you can use the tee option. The tee can be started with option --tee=..., or from the command line interactively with command tee. All the data displayed on the screen will also be appended into a given file. This can be very useful for debugging purposes also. The tee can be disabled from the command line with command notee. Executing tee again starts logging again. Without a parameter the previous file will be used. Note that tee will flush the results into the file after each command, just before the command line appears again waiting for the next command.
• Browsing, or searching the results in the interactive mode in UNIX less, more, or any other similar program, is now possible with option --pager[=...]. Without argument, mysql client will look for environment variable PAGER and set pager to that. pager can be started from the interactive command line with command pager and disabled with command nopager. The command takes an argument optionally and the pager will be set to that. Command pager can be called without an argument, but this requires that the option --pager was used, or the pager will default to stdout. pager works only in UNIX, since it uses the popen() function, which doesn't exist in Windows. In Windows, the tee option can be used instead, although it may not be as handy as pager can be in some situations.
• A few tips about pager: You can use it to write to a file:

  mysql> pager cat > /tmp/log.txt
  

  and the results will only go to a file. You can also pass any options for the programs that you want to use with the pager:

  mysql> pager less -n -i -S
  

  From the above do note the option '-S'. You may find it very useful when browsing the results; try the option with horizontal output (end commands with '\g', or ';') and with vertical output (end commands with '\G'). Sometimes a very wide result set is hard to be read from the screen, with option -S to less you can browse the results within the interactive less from left to right, preventing lines longer than your screen from being continued to the next line. This can make the result set much more readable. You can swith the mode between on and off within the interactive less with '-S'. See the 'h' for more help about less.
• Last (unless you already understood this from the above examples ;) you can combine very complex ways to handle the results, for example the following would send the results to two files in two different directories, on two different hard-disks mounted on /dr1 and /dr2, yet let the results still be seen on the screen via less:

  mysql> pager cat | tee /dr1/tmp/res.txt | tee /dr2/tmp/res2.txt | less -n -i -S
  

• You can also combine the two functions above; have the tee enabled, pager set to 'less' and you will be able to browse the results in unix 'less' and still have everything appended into a file the same time. The difference between UNIX tee used with the pager and the mysql client in-built tee, is that the in-built tee works even if you don't have the UNIX tee available. The in-built tee also logs everything that is printed on the screen, where the UNIX tee used with pager doesn't log quite that much. Last, but not least, the interactive tee is more handy to switch on and off, when you want to log something into a file, but want to be able to turn the feature off sometimes.

4.8.3 mysqladmin, Administrating a MySQL Server

A utility for performing administrative operations. The syntax is:

shell> mysqladmin [OPTIONS] command [command-option] command ...

You can get a list of the options your version of mysqladmin supports by executing mysqladmin --help.

The current mysqladmin supports the following commands:

create databasename

Create a new database.

drop databasename

Delete a database and all its tables.

extended-status

Gives an extended status message from the server.

flush-hosts

Flush all cached hosts.

flush-logs

Flush all logs.

flush-tables

Flush all tables.

flush-privileges

Reload grant tables (same as reload).

kill id,id,...

Kill mysql threads.

password

Set a new password. Change old password to new-password.

ping

Check if mysqld is alive.

processlist

Show list of active threads in server.

reload

Reload grant tables.

refresh

Flush all tables and close and open logfiles.

shutdown

Take server down.

slave-start

Start slave replication thread.

slave-stop

Stop slave replication thread.

status

Gives a short status message from the server.

variables

Prints variables available.

version

Get version info from server.

All commands can be shortened to their unique prefix. For example:

shell> mysqladmin proc stat
+----+-------+-----------+----+-------------+------+-------+------+
| Id | User  | Host      | db | Command     | Time | State | Info |
+----+-------+-----------+----+-------------+------+-------+------+
| 6  | monty | localhost |    | Processlist | 0    |       |      |
+----+-------+-----------+----+-------------+------+-------+------+
Uptime: 10077  Threads: 1  Questions: 9  Slow queries: 0  Opens: 6  Flush tables: 1  Open tables: 2  Memory in use: 1092K  Max memory used: 1116K

The mysqladmin status command result has the following columns:

Uptime	Number of seconds the MySQL server has been up.
Threads	Number of active threads (clients).
Questions	Number of questions from clients since mysqld was started.
Slow queries	Queries that have taken more than long_query_time seconds. See section 4.9.5 The Slow Query Log.
Opens	How many tables mysqld has opened.
Flush tables	Number of flush ..., refresh, and reload commands.
Open tables	Number of tables that are open now.
Memory in use	Memory allocated directly by the mysqld code (only available when MySQL is compiled with --with-debug=full).
Max memory used	Maximum memory allocated directly by the mysqld code (only available when MySQL is compiled with --with-debug=full).

If you do myslqadmin shutdown on a socket (in other words, on a the computer where mysqld is running), mysqladmin will wait until the MySQL pid-file is removed to ensure that the mysqld server has stopped properly.

4.8.4 Using mysqlcheck for Table Maintenance and Crash Recovery

Since MySQL version 3.23.38 you will be able to use a new checking and repairing tool for MyISAM tables. The difference to myisamchk is that mysqlcheck should be used when the mysqld server is running, where as myisamchk should be used when it is not. The benefit is that you no longer have to take the server down for checking or repairing your tables.

mysqlcheck uses MySQL server commands CHECK, REPAIR, ANALYZE and OPTIMIZE in a convenient way for the user.

There are three alternative ways to invoke mysqlcheck:

shell> mysqlcheck [OPTIONS] database [tables]
shell> mysqlcheck [OPTIONS] --databases DB1 [DB2 DB3...]
shell> mysqlcheck [OPTIONS] --all-databases

So it can be used in a similar way as mysqldump when it comes to what databases and tables you want to choose.

mysqlcheck does have a special feature compared to the other clients; the default behavior, checking tables (-c), can be changed by renaming the binary. So if you want to have a tool that repairs tables by default, you should just copy mysqlcheck to your harddrive with a new name, mysqlrepair, or alternatively make a symbolic link to mysqlrepair and name the symbolic link as mysqlrepair. If you invoke mysqlrepair now, it will repair tables by default.

The names that you can use to change mysqlcheck default behavior are here:

mysqlrepair:   The default option will be -r
mysqlanalyze:  The default option will be -a
mysqloptimize: The default option will be -o

The options available for mysqlcheck are listed here, please check what your version supports with mysqlcheck --help.

-A, --all-databases

Check all the databases. This will be same as --databases with all databases selected

-1, --all-in-1

Instead of making one query for each table, execute all queries in 1 query separately for each database. Table names will be in a comma separated list.

-a, --analyze

Analyze given tables.

--auto-repair

If a checked table is corrupted, automatically fix it. Repairing will be done after all tables have been checked, if corrupted ones were found.

-#, --debug=...

Output debug log. Often this is 'd:t:o,filename'

--character-sets-dir=...

Directory where character sets are

-c, --check

Check table for errors

-C, --check-only-changed

Check only tables that have changed since last check or haven't been closed properly.

--compress

Use compression in server/client protocol.

-?, --help

Display this help message and exit.

-B, --databases

To check several databases. Note the difference in usage; In this case no tables are given. All name arguments are regarded as database names.

--default-character-set=...

Set the default character set

-F, --fast

Check only tables that hasn't been closed properly

-f, --force

Continue even if we get an sql-error.

-e, --extended

If you are using this option with CHECK TABLE, it will ensure that the table is 100 percent consistent, but will take a long time. If you are using this option with REPAIR TABLE, it will run an extended repair on the table, which may not only take a long time to execute, but may produce a lot of garbage rows also!

-h, --host=...

Connect to host.

-m, --medium-check

Faster than extended-check, but only finds 99.99 percent of all errors. Should be good enough for most cases.

-o, --optimize

Optimize table

-p, --password[=...]

Password to use when connecting to server. If password is not given it's solicited on the tty.

-P, --port=...

Port number to use for connection.

-q, --quick

If you are using this option with CHECK TABLE, it prevents the check from scanning the rows to check for wrong links. This is the fastest check. If you are using this option with REPAIR TABLE, it will try to repair only the index tree. This is the fastest repair method for a table.

-r, --repair

Can fix almost anything except unique keys that aren't unique.

-s, --silent

Print only error messages.

-S, --socket=...

Socket file to use for connection.

--tables

Overrides option --databases (-B).

-u, --user=#

User for login if not current user.

-v, --verbose

Print info about the various stages.

-V, --version

Output version information and exit.

4.8.5 mysqldump, Dumping Table Structure and Data

Utility to dump a database or a collection of database for backup or for transferring the data to another SQL server (not necessarily a MySQL server). The dump will contain SQL statements to create the table and/or populate the table.

If you are doing a backup on the server, you should consider using the mysqlhotcopy instead. See section 4.8.6 mysqlhotcopy, Copying MySQL Databases and Tables.

shell> mysqldump [OPTIONS] database [tables]
OR     mysqldump [OPTIONS] --databases [OPTIONS] DB1 [DB2 DB3...]
OR     mysqldump [OPTIONS] --all-databases [OPTIONS]

If you don't give any tables or use the --databases or --all-databases, the whole database(s) will be dumped.

You can get a list of the options your version of mysqldump supports by executing mysqldump --help.

Note that if you run mysqldump without --quick or --opt, mysqldump will load the whole result set into memory before dumping the result. This will probably be a problem if you are dumping a big database.

Note that if you are using a new copy of the mysqldump program and you are going to do a dump that will be read into a very old MySQL server, you should not use the --opt or -e options.

mysqldump supports the following options:

--add-locks

Add LOCK TABLES before and UNLOCK TABLE after each table dump. (To get faster inserts into MySQL.)

--add-drop-table

Add a drop table before each create statement.

-A, --all-databases

Dump all the databases. This will be same as --databases with all databases selected.

-a, --all

Include all MySQL-specific create options.

--allow-keywords

Allow creation of column names that are keywords. This works by prefixing each column name with the table name.

-c, --complete-insert

Use complete insert statements (with column names).

-C, --compress

Compress all information between the client and the server if both support compression.

-B, --databases

To dump several databases. Note the difference in usage. In this case no tables are given. All name arguments are regarded as database names. USE db_name; will be included in the output before each new database.

--delayed

Insert rows with the INSERT DELAYED command.

-e, --extended-insert

Use the new multiline INSERT syntax. (Gives more compact and faster inserts statements.)

-#, --debug[=option_string]

Trace usage of the program (for debugging).

--help

Display a help message and exit.

--fields-terminated-by=...

--fields-enclosed-by=...

--fields-optionally-enclosed-by=...

--fields-escaped-by=...

--lines-terminated-by=...

These options are used with the -T option and have the same meaning as the corresponding clauses for LOAD DATA INFILE. See section 6.4.9 LOAD DATA INFILE Syntax.

-F, --flush-logs

Flush log file in the MySQL server before starting the dump.

-f, --force,

Continue even if we get a SQL error during a table dump.

-h, --host=..

Dump data from the MySQL server on the named host. The default host is localhost.

-l, --lock-tables.

Lock all tables before starting the dump. The tables are locked with READ LOCAL to allow concurrent inserts in the case of MyISAM tables.

-n, --no-create-db

'CREATE DATABASE /*!32312 IF NOT EXISTS*/ db_name;' will not be put in the output. The above line will be added otherwise, if --databases or --all-databases option was given.

-t, --no-create-info

Don't write table creation information (The CREATE TABLE statement.)

-d, --no-data

Don't write any row information for the table. This is very useful if you just want to get a dump of the structure for a table!

--opt

Same as --quick --add-drop-table --add-locks --extended-insert --lock-tables. Should give you the fastest possible dump for reading into a MySQL server.

-pyour_pass, --password[=your_pass]

The password to use when connecting to the server. If you specify no `=your_pass' part, mysqldump you will be prompted for a password.

-P port_num, --port=port_num

The TCP/IP port number to use for connecting to a host. (This is used for connections to hosts other than localhost, for which Unix sockets are used.)

-q, --quick

Don't buffer query, dump directly to stdout. Uses mysql_use_result() to do this.

-r, --result-file=...

Direct output to a given file. This option should be used in MSDOS, because it prevents new line '\n' from being converted to '\n\r' (new line + carriage return).

-S /path/to/socket, --socket=/path/to/socket

The socket file to use when connecting to localhost (which is the default host).

--tables

Overrides option --databases (-B).

-T, --tab=path-to-some-directory

Creates a table_name.sql file, that contains the SQL CREATE commands, and a table_name.txt file, that contains the data, for each give table. NOTE: This only works if mysqldump is run on the same machine as the mysqld daemon. The format of the .txt file is made according to the --fields-xxx and --lines--xxx options.

-u user_name, --user=user_name

The MySQL user name to use when connecting to the server. The default value is your Unix login name.

-O var=option, --set-variable var=option

Set the value of a variable. The possible variables are listed below.

-v, --verbose

Verbose mode. Print out more information on what the program does.

-V, --version

Print version information and exit.

-w, --where='where-condition'

Dump only selected records. Note that QUOTES are mandatory:

"--where=user='jimf'" "-wuserid>1" "-wuserid<1"

-O net_buffer_length=#, where # < 16M

When creating multi-row-insert statements (as with option --extended-insert or --opt), mysqldump will create rows up to net_buffer_length length. If you increase this variable, you should also ensure that the max_allowed_packet variable in the MySQL server is bigger than the net_buffer_length.

The most normal use of mysqldump is probably for making a backup of whole databases. See section 4.4.1 Database Backups.

mysqldump --opt database > backup-file.sql

You can read this back into MySQL with:

mysql database < backup-file.sql

or

mysql -e "source /patch-to-backup/backup-file.sql" database

However, it's also very useful to populate another MySQL server with information from a database:

mysqldump --opt database | mysql --host=remote-host -C database

It is possible to dump several databases with one command:

mysqldump --databases database1 [database2 database3...] > my_databases.sql

If all the databases are wanted, one can use:

mysqldump --all-databases > all_databases.sql

4.8.6 mysqlhotcopy, Copying MySQL Databases and Tables

mysqlhotcopy is a perl script that uses LOCK TABLES, FLUSH TABLES and cp or scp to quickly make a backup of a database. It's the fastest way to make a backup of the database, of single tables but it can only be run on the same machine where the database directories are.

mysqlhotcopy db_name [/path/to/new_directory]

mysqlhotcopy db_name_1 ... db_name_n /path/to/new_directory

mysqlhotcopy db_name./regex/

mysqlhotcopy supports the following options:

-?, --help

Display a help screen and exit

-u, --user=#

User for database login

-p, --password=#

Password to use when connecting to server

-P, --port=#

Port to use when connecting to local server

-S, --socket=#

Socket to use when connecting to local server

--allowold

Don't abort if target already exists (rename it _old)

--keepold

Don't delete previous (now renamed) target when done

--noindices

Don't include full index files in copy to make the backup smaller and faster The indexes can later be reconstructed with myisamchk -rq..

--method=#

Method for copy (cp or scp).

-q, --quiet

Be silent except for errors

--debug

Enable debug

-n, --dryrun

Report actions without doing them

--regexp=#

Copy all databases with names matching regexp

--suffix=#

Suffix for names of copied databases

--checkpoint=#

Insert checkpoint entry into specified db.table

--flushlog

Flush logs once all tables are locked.

--tmpdir=#

Temporary directory (instead of /tmp).

You can use perldoc mysqlhotcopy to get a more complete documentation for mysqlhotcopy.

mysqlhotcopy reads the groups [client] and [mysqlhotcopy] from the option files.

To be able to execute mysqlhotcopy you need write access to the backup directory, SELECT privilege to the tables you are about to copy and the MySQL Reload privilege (to be able to execute FLUSH TABLES).

4.8.7 mysqlimport, Importing Data from Text Files

mysqlimport provides a command-line interface to the LOAD DATA INFILE SQL statement. Most options to mysqlimport correspond directly to the same options to LOAD DATA INFILE. See section 6.4.9 LOAD DATA INFILE Syntax.

mysqlimport is invoked like this:

shell> mysqlimport [options] database textfile1 [textfile2....]

For each text file named on the command line, mysqlimport strips any extension from the filename and uses the result to determine which table to import the file's contents into. For example, files named `patient.txt', `patient.text', and `patient' would all be imported into a table named patient.

mysqlimport supports the following options:

-c, --columns=...

This option takes a comma-separated list of field names as an argument. The field list is used to create a proper LOAD DATA INFILE command, which is then passed to MySQL. See section 6.4.9 LOAD DATA INFILE Syntax.

-C, --compress

Compress all information between the client and the server if both support compression.

-#, --debug[=option_string]

Trace usage of the program (for debugging).

-d, --delete

Empty the table before importing the text file.

--fields-terminated-by=...

--fields-enclosed-by=...

--fields-optionally-enclosed-by=...

--fields-escaped-by=...

--lines-terminated-by=...

These options have the same meaning as the corresponding clauses for LOAD DATA INFILE. See section 6.4.9 LOAD DATA INFILE Syntax.

-f, --force

Ignore errors. For example, if a table for a text file doesn't exist, continue processing any remaining files. Without --force, mysqlimport exits if a table doesn't exist.

--help

Display a help message and exit.

-h host_name, --host=host_name

Import data to the MySQL server on the named host. The default host is localhost.

-i, --ignore

See the description for the --replace option.

-l, --lock-tables

Lock ALL tables for writing before processing any text files. This ensures that all tables are synchronized on the server.

-L, --local

Read input files from the client. By default, text files are assumed to be on the server if you connect to localhost (which is the default host).

-pyour_pass, --password[=your_pass]

The password to use when connecting to the server. If you specify no `=your_pass' part, mysqlimport you will be prompted for a password.

-P port_num, --port=port_num

The TCP/IP port number to use for connecting to a host. (This is used for connections to hosts other than localhost, for which Unix sockets are used.)

-r, --replace

The --replace and --ignore options control handling of input records that duplicate existing records on unique key values. If you specify --replace, new rows replace existing rows that have the same unique key value. If you specify --ignore, input rows that duplicate an existing row on a unique key value are skipped. If you don't specify either option, an error occurs when a duplicate key value is found, and the rest of the text file is ignored.

-s, --silent

Silent mode. Write output only when errors occur.

-S /path/to/socket, --socket=/path/to/socket

The socket file to use when connecting to localhost (which is the default host).

-u user_name, --user=user_name

The MySQL user name to use when connecting to the server. The default value is your Unix login name.

-v, --verbose

Verbose mode. Print out more information what the program does.

-V, --version

Print version information and exit.

Here is a sample run using mysqlimport:

$ mysql --version
mysql  Ver 9.33 Distrib 3.22.25, for pc-linux-gnu (i686)
$ uname -a
Linux xxx.com 2.2.5-15 #1 Mon Apr 19 22:21:09 EDT 1999 i586 unknown
$ mysql -e 'CREATE TABLE imptest(id INT, n VARCHAR(30))' test
$ ed
a
100     Max Sydow
101     Count Dracula
.
w imptest.txt
32
q
$ od -c imptest.txt
0000000   1   0   0  \t   M   a   x       S   y   d   o   w  \n   1   0
0000020   1  \t   C   o   u   n   t       D   r   a   c   u   l   a  \n
0000040
$ mysqlimport --local test imptest.txt
test.imptest: Records: 2  Deleted: 0  Skipped: 0  Warnings: 0
$ mysql -e 'SELECT * FROM imptest' test
+------+---------------+
| id   | n             |
+------+---------------+
|  100 | Max Sydow     |
|  101 | Count Dracula |
+------+---------------+

4.8.8 Showing Databases, Tables, and Columns

mysqlshow can be used to quickly look at which databases exist, their tables, and the table's columns.

With the mysql program you can get the same information with the SHOW commands. See section 4.5.5 SHOW Syntax.

mysqlshow is invoked like this:

shell> mysqlshow [OPTIONS] [database [table [column]]]

• If no database is given, all matching databases are shown.
• If no table is given, all matching tables in the database are shown.
• If no column is given, all matching columns and column types in the table are shown.

Note that in newer MySQL versions, you only see those database/tables/columns for which you have some privileges.

If the last argument contains a shell or SQL wild-card (*, ?, % or _) then only what's matched by the wild card is shown. This may cause some confusion when you try to display the columns for a table with a _ as in this case mysqlshow only shows you the table names that match the pattern. This is easily fixed by adding an extra % last on the command line (as a separate argument).

4.8.9 perror, Explaining Error Codes

perror can be used to print error message(s). perror can be invoked like this:

shell> perror [OPTIONS] [ERRORCODE [ERRORCODE...]]

For example:

shell> perror 64 79
Error code  64:  Machine is not on the network
Error code  79:  Can not access a needed shared library

perror can be used to display a description for a system error code, or an MyISAM/ISAM table handler error code. The error messages are mostly system dependent.

4.8.10 How to Run SQL Commands from a Text File

The mysql client typically is used interactively, like this:

shell> mysql database

However, it's also possible to put your SQL commands in a file and tell mysql to read its input from that file. To do so, create a text file `text_file' that contains the commands you wish to execute. Then invoke mysql as shown below:

shell> mysql database < text_file

You can also start your text file with a USE db_name statement. In this case, it is unnecessary to specify the database name on the command line:

shell> mysql < text_file

See section 4.8 MySQL Client-Side Scripts and Utilities.

4.9 The MySQL Log Files

MySQL has several different log files that can help you find out what's going on inside mysqld:

All logs can be found in the mysqld data directory. You can force mysqld to reopen the log files (or in some cases switch to a new log) by executing FLUSH LOGS. See section 4.5.3 FLUSH Syntax.

4.9.1 The Error Log

mysqld writes all errors to the stderr, which the safe_mysqld script redirects to a file called 'hostname'.err. (On Windows, mysqld writes this directly to `\mysql\data\mysql.err').

This contains information indicating when mysqld was started and stopped and also any critical errors found when running. If mysqld dies unexpectedly and safe_mysqld needs to restart mysqld, safe_mysqld will write a restarted mysqld row in this file. This log also holds a warning if mysqld notices a table that needs to be automatically checked or repaired.

On some operating systems, the error log will contain a stack trace for where mysqld died. This can be used to find out where mysqld died. See section G.1.4 Using a stack trace.

4.9.2 The General Query Log

If you want to know what happens within mysqld, you should start it with --log[=file]. This will log all connections and queries to the log file (by default named `'hostname'.log'). This log can be very useful when you suspect an error in a client and want to know exactly what mysqld thought the client sent to it.

By default, the mysql.server script starts the MySQL server with the -l option. If you need better performance when you start using MySQL in a production environment, you can remove the -l option from mysql.server or change it to --log-binary.

The entries in this log are written as mysqld receives the questions. This may be different than the order in which the statements are executed. This is in contrast to the update log and the binary log which are written after the query is executed, but before any locks are released.

4.9.3 The Update Log

NOTE: The update log is replaced by the binary log. See section 4.9.4 The Binary Update Log. With this you can do anything that you can do with the update log.

When started with the --log-update[=file_name] option, mysqld writes a log file containing all SQL commands that update data. If no filename is given, it defaults to the name of the host machine. If a filename is given, but it doesn't contain a path, the file is written in the data directory. If `file_name' doesn't have an extension, mysqld will create log file names like so: `file_name.###', where ### is a number that is incremented each time you execute mysqladmin refresh, execute mysqladmin flush-logs, execute the FLUSH LOGS statement, or restart the server.

NOTE: For the above scheme to work, you should NOT create your own files with the same filename as the update log + some extensions that may be regarded as a number, in the directory used by the update log!

If you use the --log or -l options, mysqld writes a general log with a filename of `hostname.log', and restarts and refreshes do not cause a new log file to be generated (although it is closed and reopened). In this case you can copy it (on Unix) by doing:

mv hostname.log hostname-old.log
mysqladmin flush-logs
cp hostname-old.log to-backup-directory
rm hostname-old.log

Update logging is smart because it logs only statements that really update data. So an UPDATE or a DELETE with a WHERE that finds no rows is not written to the log. It even skips UPDATE statements that set a column to the value it already has.

The update logging is done immediately after a query completes but before any locks are released or any commit is done. This ensures that the log will be logged in the execution order.

If you want to update a database from update log files, you could do the following (assuming your update logs have names of the form `file_name.###'):

shell> ls -1 -t -r file_name.[0-9]* | xargs cat | mysql

ls is used to get all the log files in the right order.

This can be useful if you have to revert to backup files after a crash and you want to redo the updates that occurred between the time of the backup and the crash.

4.9.4 The Binary Update Log

In the future the binary log will replace the update log, so we recommend you to switch to this log format as soon as possible!

The binary log contains all information that is available in the update log in a more efficient format. It also contains information about how long every query that updated the database took.

The binary log is also used when you are replicating a slave from a master. See section 4.10 Replication in MySQL.

When started with the --log-bin[=file_name] option, mysqld writes a log file containing all SQL commands that update data. If no file name is given, it defaults to the name of the host machine followed by -bin. If file name is given, but it doesn't contain a path, the file is written in the data directory.

You can use the following options to mysqld to affect what is logged to the binary log:

To the binary log filename mysqld will append an extension that is a number that is incremented each time you execute mysqladmin refresh, execute mysqladmin flush-logs, execute the FLUSH LOGS statement or restart the server.

To be able to know which different binary log files have been used, mysqld will also create a binary log index file that contains the name of all used binary log files. By default this has the same name as the binary log file, with the extension '.index'. You can change the name of the binary log index file with the --log-bin-index=[filename] option.

If you are using replication, you should not delete old binary log files until you are sure that no slave will ever need to use them. One way to do this is to do mysqladmin flush-logs once a day and then remove any logs that are more than 3 days old.

You can examine the binary log file with the mysqlbinlog command. For example, you can update a MySQL server from the binary log as follows:

mysqlbinlog log-file | mysql -h server_name

You can also use the mysqlbinlog program to read the binary log directly from a remote MySQL server!

mysqlbinlog --help will give you more information of how to use this program!

If you are using BEGIN [WORK] or SET AUTOCOMMIT=0, you must use the MySQL binary log for backups instead of the old update log.

The binary logging is done immediately after a query completes but before any locks are released or any commit is done. This ensures that the log will be logged in the execution order.

All updates (UPDATE, DELETE or INSERT) that change a transactional table (like BDB tables) are cached until a COMMIT. Any updates to a non-transactional table are stored in the binary log at once. Every thread will, on start, allocate a buffer of binlog_cache_size to buffer queries. If a query is bigger than this, the thread will open a temporary file to handle the bigger cache. The temporary file will be deleted when the thread ends.

The max_binlog_cache_size can be used to restrict the total size used to cache a multi-transaction query.

If you are using the update or binary log, concurrent inserts will not work together with CREATE ... INSERT and INSERT ... SELECT. This is to ensure that you can recreate an exact copy of your tables by applying the log on a backup.

4.9.5 The Slow Query Log

When started with the --log-slow-queries[=file_name] option, mysqld writes a log file containing all SQL commands that took more than long_query_time to execute. The time to get the initial table locks are not counted as execution time.

The slow query log is logged after the query is executed and after all locks has been released. This may be different than the order in which the statements are executed.

If no file name is given, it defaults to the name of the host machine suffixed with -slow.log. If a filename is given, but doesn't contain a path, the file is written in the data directory.

The slow query log can be used to find queries that take a long time to execute and are thus candidates for optimization. With a large log, that can become a difficult task. You can pipe the slow query log through the mysqldumpslow command to get a summary of the queries which appear in the log.

You are using --log-long-format then also queries that are not using indexes are printed. See section 4.1.1 mysqld Command-line Options.

4.9.6 Log File Maintenance

MySQL has a lot of log files which make it easy to see what is going. See section 4.9 The MySQL Log Files. One must however from time to time clean up after MysQL to ensure that the logs don't take up too much disk space.

When using MySQL with log files, you will, from time to time, want to remove/backup old log files and tell MySQL to start logging on new files. See section 4.4.1 Database Backups.

On a Linux (Redhat) installation, you can use the mysql-log-rotate script for this. If you installed MySQL from an RPM distribution, the script should have been installed automatically. Note that you should be careful with this if you are using the log for replication!

On other systems you must install a short script yourself that you start from cron to handle log files.

You can force MySQL to start using new log files by using mysqladmin flush-logs or by using the SQL command FLUSH LOGS. If you are using MySQL Version 3.21 you must use mysqladmin refresh.

The above command does the following:

• If standard logging (--log) or slow query logging (--log-slow-queries) is used, closes and reopens the log file. (`mysql.log' and ``hostname`-slow.log' as default).
• If update logging (--log-update) is used, closes the update log and opens a new log file with a higher sequence number.

If you are using only an update log, you only have to flush the logs and then move away the old update log files to a backup. If you are using the normal logging, you can do something like:

shell> cd mysql-data-directory
shell> mv mysql.log mysql.old
shell> mysqladmin flush-logs

and then take a backup and remove `mysql.old'.

4.10 Replication in MySQL

This chapter describes the various replication features in MySQL. It serves as a reference to the options available with replication. You will be introduced to replication and learn how to implement it. Towards the end, there are some frequently asked questions and descriptions of problems and how to solve them.

4.10.1 Introduction

One way replication can be used is to increase both robustness and speed. For robustness you can have two systems and can switch to the backup if you have problems with the master. The extra speed is achieved by sending a part of the non-updating queries to the replica server. Of course this only works if non-updating queries dominate, but that is the normal case.

Starting in Version 3.23.15, MySQL supports one-way replication internally. One server acts as the master, while the other acts as the slave. Note that one server could play the roles of master in one pair and slave in the other. The master server keeps a binary log of updates (See section 4.9.4 The Binary Update Log.) and an index file to binary logs to keep track of log rotation. The slave, upon connecting, informs the master where it left off since the last successfully propagated update, catches up on the updates, and then blocks and waits for the master to notify it of the new updates.

Note that if you are replicating a database, all updates to this database should be done through the master!

Another benefit of using replication is that one can get live backups of the system by doing a backup on a slave instead of doing it on the master. See section 4.4.1 Database Backups.

4.10.2 Replication Implementation Overview

MySQL replication is based on the server keeping track of all changes to your database (updates, deletes, etc) in the binary log. (See section 4.9.4 The Binary Update Log.) and the slave server(s) reading the saved queries from the master server's binary log so that the slave can execute the same queries on its copy of the data.

It is very important to realize that the binary log is simply a record starting from a fixed point in time (the moment you enable binary logging). Any slaves which you set up will need copies of all the data from your master as it existed the moment that you enabled binary logging on the master. If you start your slaves with data that doesn't agree with what was on the master when the binary log was started, your slaves may fail.

A future version (4.0) of MySQL will remove the need to keep a (possibly large) snapshot of data for new slaves that you might wish to set up through the live backup functionality with no locking required. However, at this time, it is necessary to block all writes either with a global read lock or by shutting down the master while taking a snapshot.

Once a slave is properly configured and running, it will simply connect to the master and wait for updates to process. If the master goes away or the slave loses connectivity with your master, it will keep trying to connect every master-connect-retry seconds until it is able to reconnect and resume listening for updates.

Each slave keeps track of where it left off. The master server has no knowledge of how many slaves there are or which ones are up-to-date at any given time.

The next section explains the master/slave setup process in more detail.

4.10.3 How To Set Up Replication

Below is a quick description of how to set up complete replication on your current MySQL server. It assumes you want to replicate all your databases and have not configured replication before. You will need to shutdown your master server briefly to complete the steps outlined below.

1. Make sure you have a recent version of MySQL installed on the master and slave(s). Use Version 3.23.29 or higher. Previous releases used a different binary log format and had bugs which have been fixed in newer releases. Please, do not report bugs until you have verified that the problem is present in the latest release.
2. Set up special a replication user on the master with the FILE privilege and permission to connect from all the slaves. If the user is only doing replication (which is recommended), you don't need to grant any additional privileges. For example, to create a user named repl which can access your master from any host, you might use this command:

   GRANT FILE ON *.* TO repl@"%" IDENTIFIED BY '<password>';
   

3. Shut down MySQL on the master.

   mysqladmin -u root -p<password> shutdown
   

4. Snapshot all the data on your master server. The easiest way to do this (on Unix) is to simply use tar to produce an archive of your entire data directory. The exact data directory location depends on your installation.

   tar -cvf /tmp/mysql-snapshot.tar /path/to/data-dir
   

   Windows users can use WinZip or similar software to create an archive of the data directory.
5. In my.cnf on the master add log-bin and server-id=unique number to the [mysqld] section and restart it. It is very important that the id of the slave is different from the id of the master. Think of server-id as something similar to the IP address - it uniquely identifies the server instance in the community of replication partners.

   [mysqld]
   log-bin
   server-id=1
   

6. Restart MySQL on the master.
7. Add the following to my.cnf on the slave(s):

   master-host=<hostname of the master>
   master-user=<replication user name>
   master-password=<replication user password>
   master-port=<TCP/IP port for master>
   server-id=<some unique number between 2 and 2^32-1>
   

   replacing the values in <> with what is relevant to your system. server-id must be different for each server participating in replication. If you don't specify a server-id, it will be set to 1 if you have not defined master-host, else it will be set to 2. Note that in the case of server-id omission the master will refuse connections from all slaves, and the slave will refuse to connect to a master. Thus, omitting server-id is only good for backup with a binary log.
8. Copy the snapshot data into your data directory on your slave(s). Make sure that the privileges on the files and directories are correct. The user which MySQL runs as needs to be able to read and write to them, just as on the master.
9. Restart the slave(s).

After you have done the above, the slave(s) should connect to the master and catch up on any updates which happened since the snapshot was taken.

If you have forgotten to set server-id for the slave you will get the following error in the error log file:

Warning: one should set server_id to a non-0 value if master_host is set.
The server will not act as a slave.

If you have forgot to do this for the master, the slaves will not be able to connect to the master.

If a slave is not able to replicate for any reason, you will find error messages in the error log on the slave.

Once a slave is replicating, you will find a file called master.info in the same directory as your error log. The master.info file is used by the slave to keep track of how much of the master's binary log is has processed. Do not remove or edit the file, unless you really know what you are doing. Even in that case, it is preferred that you use CHANGE MASTER TO command.

4.10.4 Replication Features and Known Problems

Below is an explanation of what is supported and what is not:

• Replication will be done correctly with AUTO_INCREMENT, LAST_INSERT_ID, and TIMESTAMP values.
• RAND() in updates does not replicate properly. Use RAND(some_non_rand_expr) if you are replicating updates with RAND(). You can, for example, use UNIX_TIMESTAMP() for the argument to RAND().
• You have to use the same character set (--default-character-set) on the master and the slave. If not, you may get duplicate key errors on the slave, because a key that is regarded as unique on the master may not be that in the other character set.
• LOAD DATA INFILE will be handled properly as long as the file still resides on the master server at the time of update propagation. LOAD LOCAL DATA INFILE will be skipped.
• Update queries that use user variables are not replication-safe (yet).
• FLUSH commands are not stored in the binary log and are because of this not replicated to the slaves. This is not normally a problem as FLUSH doesn't change anything. This does however mean that if you update the MySQL privilege tables directly without using GRANT statement and you replicate the MySQL privilege database, you must do a FLUSH PRIVILEGES on your slaves to put the new privileges into effect.
• Temporary tables starting in 3.23.29 are replicated properly with the exception of the case when you shut down slave server ( not just slave thread), you have some temporary tables open, and the are used in subsequent updates. To deal with this problem, to shut down the slave, do SLAVE STOP, then check Slave_open_temp_tables variable to see if it is 0, then issue mysqladmin shutdown. If the number is not 0, restart the slave thread with SLAVE START and see if you have better luck next time. There will be a cleaner solution, but it has to wait until version 4.0. In earlier versions temporary tables are not being replicated properly - we recommend that you either upgrade, or execute SET SQL_LOG_BIN=0 on your clients before all queries with temp tables.
• MySQL only supports one master and many slaves. We will in 4.x add a voting algorithm to automatically change master if something goes wrong with the current master. We will also introduce 'agent' processes to help doing load balancing by sending select queries to different slaves.
• Starting in Version 3.23.26, it is safe to connect servers in a circular master-slave relationship with log-slave-updates enabled. Note, however, that many queries will not work right in this kind of setup unless your client code is written to take care of the potential problems that can happen from updates that occur in different sequence on different servers. This means that you can do a setup like the following:

  A -> B -> C -> A
  

  This setup will only works if you only do non conflicting updates between the tables. In other words, if you insert data in A and C, you should never insert a row in A that may have a conflicting key with a row insert in C. You should also not update the sam rows on two servers if the order in which the updates are applied matters. Note that the log format has changed in Version 3.23.26 so that pre-3.23.26 slaves will not be able to read it.
• If the query on the slave gets an error, the slave thread will terminate, and a message will appear in the .err file. You should then connect to the slave manually, fix the cause of the error (for example, non-existent table), and then run SLAVE START sql command (available starting in Version 3.23.16). In Version 3.23.15, you will have to restart the server.
• If connection to the master is lost, the slave will retry immediately, and then in case of failure every master-connect-retry (default 60) seconds. Because of this, it is safe to shut down the master, and then restart it after a while. The slave will also be able to deal with network connectivity outages.
• Shutting down the slave (cleanly) is also safe, as it keeps track of where it left off. Unclean shutdowns might produce problems, especially if disk cache was not synced before the system died. Your system fault tolerance will be greatly increased if you have a good UPS.
• If the master is listening on a non-standard port, you will also need to specify this with master-port parameter in my.cnf .
• In Version 3.23.15, all of the tables and databases will be replicated. Starting in Version 3.23.16, you can restrict replication to a set of databases with replicate-do-db directives in my.cnf or just exclude a set of databases with replicate-ignore-db. Note that up until Version 3.23.23, there was a bug that did not properly deal with LOAD DATA INFILE if you did it in a database that was excluded from replication.
• Starting in Version 3.23.16, SET SQL_LOG_BIN = 0 will turn off replication (binary) logging on the master, and SET SQL_LOG_BIN = 1 will turn in back on - you must have the process privilege to do this.
• Starting in Version 3.23.19, you can clean up stale replication leftovers when something goes wrong and you want a clean start with FLUSH MASTER and FLUSH SLAVE commands. In Version 3.23.26 we have renamed them to RESET MASTER and RESET SLAVE respectively to clarify what they do. The old FLUSH variants still work, though, for compatibility.
• Starting in Version 3.23.21, you can use LOAD TABLE FROM MASTER for network backup and to set up replication initially. We have recently received a number of bug reports concerning it that we are investigating, so we recommend that you use it only in testing until we make it more stable.
• Starting in Version 3.23.23, you can change masters and adjust log position with CHANGE MASTER TO.
• Starting in Version 3.23.23, you tell the master that updates in certain databases should not be logged to the binary log with binlog-ignore-db.
• Starting in Version 3.23.26, you can use replicate-rewrite-db to tell the slave to apply updates from one database on the master to the one with a different name on the slave.
• Starting in Version 3.23.28, you can use PURGE MASTER LOGS TO 'log-name' to get rid of old logs while the slave is running.

4.10.5 Replication Options in my.cnf

If you are using replication, we recommend you to use MySQL Version 3.23.30 or later. Older versions work, but they do have some bugs and are missing some features.

On both master and slave you need to use the server-id option. This sets an unique replication id. You should pick a unique value in the range between 1 to 2^32-1 for each master and slave. Example: server-id=3

The following table has the options you can use for the MASTER:

The following table has the options you can use for the SLAVE:

4.10.6 SQL Commands Related to Replication

Replication can be controlled through the SQL interface. Below is the summary of commands:

CHANGE MASTER TO
  MASTER_HOST='master2.mycompany.com',
  MASTER_USER='replication',
  MASTER_PASSWORD='bigs3cret',
  MASTER_PORT=3306,
  MASTER_LOG_FILE='master2-bin.001',
  MASTER_LOG_POS=4;

PURGE MASTER LOGS TO 'mysql-bin.010'

4.10.7 Replication FAQ

Q: Why do I sometimes see more than one Binlog_Dump thread on the master after I have restarted the slave?

A: Binlog_Dump is a continuous process that is handled by the server in the following way:

• Catch up on the updates.
• Once there are no more updates left, go into pthread_cond_wait(), from which we can be awakened either by an update or a kill.
• On wake up, check the reason. If we are not supposed to die, continue the Binlog_dump loop.
• If there is some fatal error, such as detecting a dead client, terminate the loop.

So if the slave thread stops on the slave, the corresponding Binlog_Dump thread on the master will not notice it until after at least one update to the master (or a kill), which is needed to wake it up from pthread_cond_wait(). In the meantime, the slave could have opened another connection, which resulted in another Binlog_Dump thread.

The above problem should not be present in Version 3.23.26 and later versions. In Version 3.23.26 we added server-id to each replication server, and now all the old zombie threads are killed on the master when a new replication thread connects from the same slave

Q: How do I rotate replication logs?

A: In Version 3.23.28 you should use PURGE MASTER LOGS TO command after determining which logs can be deleted, and optionally backing them up first. In earlier versions the process is much more painful, and cannot be safely done without stopping all the slaves in the case that you plan to re-use log names. You will need to stop the slave threads, edit the binary log index file, delete all the old logs, restart the master, start slave threads, and then remove the old log files.

Q: How do I upgrade on a hot replication setup?

A: If you are upgrading pre-3.23.26 versions, you should just lock the master tables, let the slave catch up, then run FLUSH MASTER on the master, and FLUSH SLAVE on the slave to reset the logs, then restart new versions of the master and the slave. Note that the slave can stay down for some time - since the master is logging all the updates, the slave will be able to catch up once it is up and can connect.

After 3.23.26, we have locked the replication protocol for modifications, so you can upgrade masters and slave on the fly to a newer 3.23 version and you can have different versions of MySQL running on the slave and the master, as long as they are both newer than 3.23.26.

Q: What issues should I be aware of when setting up two-way replication?

A: MySQL replication currently does not support any locking protocol between master and slave to guarantee the atomicity of a distributed (cross-server) update. In in other words, it is possible for client A to make an update to co-master 1, and in the meantime, before it propagates to co-master 2, client B could make an update to co-master 2 that will make the update of client A work differently than it did on co-master 1. Thus when the update of client A will make it to co-master 2, it will produce tables that will be different than what you have on co-master 1, even after all the updates from co-master 2 have also propagated. So you should not co-chain two servers in a two-way replication relationship, unless you are sure that you updates can safely happen in any order, or unless you take care of mis-ordered updates somehow in the client code.

You must also realize that two-way replication actually does not improve performance very much, if at all, as far as updates are concerned. Both servers need to do the same amount of updates each, as you would have one server do. The only difference is that there will be a little less lock contention, because the updates originating on another server will be serialized in one slave thread. This benefit, though, might be offset by network delays.

Q: How can I use replication to improve performance of my system?

A: You should set up one server as the master, and direct all writes to it, and configure as many slaves as you have the money and rackspace for, distributing the reads among the master and the slaves. You can also start the slaves with --skip-bdb, --low-priority-updates and --delay-key-write-for-all-tables to get speed improvements for the slave. In this case the slave will use non-transactional MyISAM tables instead of BDB tables to get more speed.

Q: What should I do to prepare my client code to use performance-enhancing replication?

A: If the part of your code that is responsible for database access has been properly abstracted/modularized, converting it to run with the replicated setup should be very smooth and easy - just change the implementation of your database access to read from some slave or the master, and to always write to the master. If your code does not have this level of abstraction, setting up a replicated system will give you an opportunity/motivation to it clean up. You should start by creating a wrapper library /module with the following functions:

• safe_writer_connect()
• safe_reader_connect()
• safe_reader_query()
• safe_writer_query()

safe_ means that the function will take care of handling all the error conditions.

You should then convert your client code to use the wrapper library. It may be a painful and scary process at first, but it will pay off in the long run. All applications that follow the above pattern will be able to take advantage of one-master/many slaves solution. The code will be a lot easier to maintain, and adding troubleshooting options will be trivial. You will just need to modify one or two functions, for example, to log how long each query took, or which query, among your many thousands, gave you an error. If you have written a lot of code already, you may want to automate the conversion task by using Monty's replace utility, which comes with the standard distribution of MySQL, or just write your own Perl script. Hopefully, your code follows some recognizable pattern. If not, then you are probably better off re-writing it anyway, or at least going through and manually beating it into a pattern.

Note that, of course, you can use different names for the functions. What is important is having unified interface for connecting for reads, connecting for writes, doing a read, and doing a write.

Q: When and how much can MySQL replication improve the performance of my system?

A: MySQL replication is most beneficial for a system with frequent reads and not so frequent writes. In theory, by using a one master/many slaves setup you can scale by adding more slaves until you either run out of network bandwidth, or your update load grows to the point that the master cannot handle it.

In order to determine how many slaves you can get before the added benefits begin to level out, and how much you can improve performance of your site, you need to know your query patterns, and empirically (by benchmarking) determine the relationship between the throughput on reads (reads per second, or max_reads) and on writes max_writes) on a typical master and a typical slave. The example below will show you a rather simplified calculation of what you can get with replication for our imagined system.

Let's say our system load consists of 10% writes and 90% reads, and we have determined that max_reads = 1200 - 2 * max_writes, or in other words, our system can do 1200 reads per second with no writes, our average write is twice as slow as average read, and the relationship is linear. Let us suppose that our master and slave are of the same capacity, and we have N slaves and 1 master. Then we have for each server (master or slave):

reads = 1200 - 2 * writes (from bencmarks)

reads = 9* writes / (N + 1) (reads split, but writes go to all servers)

9*writes/(N+1) + 2 * writes = 1200

writes = 1200/(2 + 9/(N+1)

So if N = 0, which means we have no replication, our system can handle 1200/11, about 109 writes per second (which means we will have 9 times as many reads due to the nature of our application).

If N = 1, we can get up to 184 writes per second.

If N = 8, we get up to 400.

If N = 17, 480 writes.

Eventually as N approaches infinity (and our budget negative infinity), we can get very close to 600 writes per second, increasing system throughput about 5.5 times. However, with only 8 servers, we increased it almost 4 times already.

Note that our computations assumed infinite network bandwidth, and neglected several other factors that could turn out to be significant on your system. In many cases, you may not be able to make a computation similar to the one above that will accurately predict what will happen on your system if you add N replication slaves. However, answering the following questions should help you decided whether and how much, if at all, the replication will improve the performance of your system:

• What is the read/write ratio on your system?
• How much more write load can one server handle if you reduce the reads?
• How many slaves do you have bandwidth for on your network?

Q: How can I use replication to provide redundancy/high availability?

A: With the currently available features, you would have to set up a master and a slave (or several slaves), and write a script that will monitor the master to see if it is up, and instruct your applications and the slaves of the master change in case of failure. Some suggestions:

• To tell a slave to change the master use the CHANGE MASTER TO command.
• A good way to keep your applications informed where the master is by having a dynamic DNS entry for the master. With bind you can use nsupdate to dynamically update your DNS.
• You should run your slaves with the log-bin option and without log-slave-updates. This way the slave will be ready to become a master as soon as you issue STOP SLAVE; RESET MASTER, and CHANGE MASTER TO on the other slaves. It will also help you catch spurious updates that may happen because of misconfiguration of the slave (ideally, you want to configure access rights so that no client can update the slave, except for the slave thread) combined with the bugs in your client programs (they should never update the slave directly).

We are currently working on integrating an automatic master election system into MySQL, but until it is ready, you will have to create your own monitoring tools.

4.10.8 Troubleshooting Replication

If you have followed the instructions, and your replication setup is not working, first eliminate the user error factor by checking the following:

• Is the master logging to the binary log? Check with SHOW MASTER STATUS. If it is, Position will be non-zero. If not, verify that you have given the master log-bin option and have set server-id.
• Is the slave running? Check with SHOW SLAVE STATUS. The answer is found in Slave_running column. If not, verify slave options and check the error log for messages.
• If the slave is running, did it establish connection with the master? Do SHOW PROCESSLIST, find the thread with system user value in User column and none in the Host column, and check the State column. If it says connecting to master, verify the privileges for the replication user on the master, master host name, your DNS setup, whether the master is actually running, whether it is reachable from the slave, and if all that seems ok, read the error logs.
• If the slave was running, but then stopped, look at SHOW SLAVE STATUS output and check the error logs. It usually happens when some query that succeeded on the master fails on the slave. This should never happen if you have taken a proper snapshot of the master, and never modify the data on the slave outside of the slave thread. If it does, it is a bug, read below on how to report it.
• If a query on that succeeded on the master refuses to run on the slave, and a full database resync ( the proper thing to do ) does not seem feasible, try the following:
  • First see if there is some stray record in the way. Understand how it got there, then delete it and run SLAVE START
  • If the above does not work or does not apply, try to understand if it would be safe to make the update manually ( if needed) and then ignore the next query from the master.
  • If you have decided you can skip the next query, do SET SQL_SLAVE_SKIP_COUNTER=1; SLAVE START; to skip a query that does not use auto_increment, or last_insert_id or SET SQL_SLAVE_SKIP_COUNTER=2; SLAVE START; otherwise. The reason auto_increment/last_insert_id queries are different is that they take two events in the binary log of the master.
  • If you are sure the slave started out perfectly in sync with the master, and no one has updated the tables involved outside of slave thread, report the bug, so you will not have to do the above tricks again.
• Make sure you are not running into an old bug by upgrading to the most recent version.
• If all else fails, read the error logs. If they are big, grep -i slave /path/to/your-log.err on the slave. There is no generic pattern to search for on the master, as the only errors it logs are general system errors - if it can, it will send the error to the slave when things go wrong.

When you have determined that there is no user error involved, and replication still either does not work at all or is unstable, it is time to start working on a bug report. We need to get as much info as possible from you to be able to track down the bug. Please do spend some time and effort preparing a good bug report. Ideally, we would like to have a test case in the format found in mysql-test/t/rpl* directory of the source tree. If you submit a test case like that, you can expect a patch within a day or two in most cases, although, of course, you mileage may vary depending on a number of factors.

Second best option is a just program with easily configurable connection arguments for the master and the slave that will demonstrate the problem on our systems. You can write one in Perl or in C, depending on which language you know better.

If you have one of the above ways to demonstrate the bug, use mysqlbug to prepare a bug report and send it to bugs@lists.mysql.com. If you have a phantom - a problem that does occur but you cannot duplicate "at will":

• Verify that there is no user error involved. For example, if you update the slave outside of the slave thread, the data will be out of sync, and you can have unique key violations on updates, in which case the slave thread will stop and wait for you to clean up the tables manually to bring them in sync.
• Run slave with log-slave-updates and log-bin - this will keep a log of all updates on the slave.
• Save all evidence before resetting the replication. If we have no or only sketchy information, it would take us a while to track down the problem. The evidence you should collect is:
  • All binary logs on the master
  • All binary log on the slave
  • The output of SHOW MASTER STATUS on the master at the time you have discovered the problem
  • The output of SHOW SLAVE STATUS on the master at the time you have discovered the problem
  • Error logs on the master and on the slave
• Use mysqlbinlog to examine the binary logs. The following should be helpful to find the trouble query, for example:

  mysqlbinlog -j pos_from_slave_status /path/to/log_from_slave_status | head
  

Once you have collected the evidence on the phantom problem, try hard to isolate it into a separate test case first. Then report the problem to bugs@lists.mysql.com with as much info as possible.

Go to the first, previous, next, last section, table of contents.

Contact us: Olivier Nicole

Last update: Nov 2001