How to run big SQL script in MySQL?

Introduction:

Use of the MySQL client can be improved at times by using script files. These files are normal text files with a .sql file extension. They can provide speed, reuse, and edibility improvements over the command line mysql client.

Requirements:

Procedure:

Script files contain any MySQL client-readable commands that could be directly invoked on the interactive client. Each statement can be separated by a line break, and terminated by semicolons (;). Script files can be used in two different ways. Within the interactive mysql client, script files can be called as any other MySQL client command would be invoked:

SOURCE filename.sql;

This script file was invoked using its relative path. For instance, if you invoked the mysql command line client from C:/Users/User/Desktop and the .sql file is located at C:/Users/User/Desktop, then the path is unnecessary, only the file name is needed. However, if you invoked the mysql client from C:/Users/User, you could locate the file by either using the relative path, or the full path:

SOURCE Desktop/filename.sql;

or the full path:

SOURCE C:/Users/User/Desktop/filename.sql;

The second approach to invoking the script file, is to invoke it from the commandine, using the mysql client but before the client is initiated with a database connection:

mysql -u root -p < C:\Users\User\Desktop\filename.sql

If the script file contains errors, the statements following the error will not be executed. To continue after the error, you can add either of the following to force continued execution:

mysql -u root -p < C:\Users\User\Desktop\filename.sql -f

or

mysql -u root -p < C:\Users\User\Desktop\filename.sql --force

Within the script file itself, you can include any MySQL client-interpretable statements, such as:

SELECT * FROM TABLENAME;

You can also include SOURCE commands inside the script file. This will call other script files for execution. Adding SOURCE calls to the script file performing the call should be avoided.

How to run big SQL script in MySQL?

Do you really need the entire database to be restored? If you don't, my 2c:

You can extract specific tables to do your restore on "chunks". Something like this:

zcat your-dump.gz.sql | sed -n -e '/DROP TABLE.*`TABLE_NAME`/,/UNLOCK TABLES/p' > table_name-dump.sql

I did it once and it took like 10 minutes to extract the table I needed - my full restore took 13~14 hours, with a 35GB (gziped) dump.

The /pattern/,/pattern/p with the -n parameter makes a slice "between the patterns" - including them.

Anyways, to restore the 35GB I used an AWS EC2 machine (c3.8xlarge), installed Percona via yum (Centos) and just added/changed the following lines on my.cnf:

max_allowed_packet=256M
wait_timeout=30000

I think the numbers are way too high, but worked for my setup.

When you need to run a saved .sql file directly from the terminal, you can use the mysql command line client.

You can run SQL scripts with or without opening a connection to the MySQL server.

First, let’s see how to run SQL files while connected to a MySQL server

Run SQL file while connected to the server

For example, suppose you have a file named main.sql with the following content:

USE school_db;
SELECT * FROM students;

The script will select a database named school_db and retrieve all rows from the students table.

To run SQL files from the terminal, you can use the source or the backslash and dot command (\.)

First, you need to connect to your MySQL database server using the mysql command. Here’s an example of connecting with the root user:

Next, enter the password for your root user.

Once inside, use the source or \. command followed by the absolute path to the SQL file as shown below:

mysql> source /Users/nsebhastian/Desktop/test/main.sql

# or

mysql> \. /Users/nsebhastian/Desktop/test/main.sql

The path /Users/nsebhastian/Desktop/test/main.sql above needs to be changed to the SQL file path on your computer.

MySQL will print the output in the command line if any. Here’s an example of running the main.sql file in my terminal:

mysql> source /Users/nsebhastian/Desktop/test/main.sql
Database changed
+----+---------------+---------+-------+--------+
| id | name          | topic   | score | gender |
+----+---------------+---------+-------+--------+
|  1 | Mark Crane    | Math    |  7.00 | male   |
|  2 | Natalia Smith | Math    |  8.00 | female |
|  3 | Gary Anderson | Math    |  0.01 | male   |
|  4 | Joe Natsume   | English |  2.50 | male   |
|  5 | Sarah         | Math    |  NULL | female |
|  6 | Peter         | English |  6.00 | male   |
|  7 | Nathan        | English |  8.00 | male   |
+----+---------------+---------+-------+--------+
7 rows in set (0.00 sec)

And that’s how you run SQL files from the terminal while being connected to MySQL database server.

Let’s see how you can run SQL files without having to connect to the server next.

Run SQL file while not connected to the server

The mysql command line client has the ability to run SQL scripts without needing to connect to MySQL database server.

You only need to provide the database_name option followed by the smaller than (<) operator as follows:

mysql -uroot -p database_name < /path/to/file.sql

For example, here’s how to run the same main.sql script without connecting to the server:

mysql -uroot -p school_db < /Users/nsebhastian/Desktop/test/main.sql

Once again, the command line client will ask for a password to run the operation.

Here’s an example of the output in my terminal:

mysql -uroot -p school_db < /Users/nsebhastian/Desktop/test/main.sql

Enter password: 

id	name	topic	score	gender
1	Mark Crane	Math	7.00	male
2	Natalia Smith	Math	8.00	female
3	Gary Anderson	Math	0.01	male
4	Joe Natsume	English	2.50	male
5	Sarah	Math	NULL	female
6	Peter	English	6.00	male
7	Nathan	English	8.00	male

As you can see from the output above, the SELECT statement result set is a bit messy compared to when you run the script while being connected to the server.

But you won’t see any difference if your SQL script contains INSERT, UPDATE, or DELETE statements.

Also, if you have a USE database_name statement in your SQL file, you can omit the database_name from the command line as shown below:

mysql -uroot -p < /Users/nsebhastian/Desktop/test/main.sql

The above example works because the main.sql file contains the USE school_db; statement.

Without the USE statement, MySQL will throw the No database selected error.

You have learned how to run SQL files or scripts from the terminal. Great job! 👍

How do I run a large SQL file in MySQL?

Save the file and run this command in mysql: set global innodb_fast_shutdown = 0. you can see the new configuration with @@ followed by command in mysql: select @@innodb_buffer_pool_size; ... .
Restart mysql: service mysql restart..
Restore your database state: mysql -u username -p database_name < /path/to/file.sql..

How do I run a large SQL script?

On the Database menu, click Execute Large Script. In Connection, select a connection to a required database server against which you want to execute your script. In Database, select a required database from the drop-down list. In File name, specify the path to the script you want to execute.

How to import large SQL file in MySQL with command line?

Show activity on this post..
Open the MySQL command line..
Type the path of your mysql bin directory and press Enter..
Paste your SQL file inside the bin folder of mysql server..
Create a database in MySQL..
Use that particular database where you want to import the SQL file..
Type source databasefilename.sql and Enter..

How to run SQL script in MySQL command line?

use the MySQL command line client: mysql -h hostname -u user database < path/to/test. sql. Install the MySQL GUI tools and open your SQL file, then execute it. Use phpmysql if the database is available via your webserver.