List of instance maintenances.
Scaleway Database is a service that make your database's management easy as pie. It provides simple and reliable solutions to create and run your database, dealing with security and high availability to ensure the durability of your data.
The following sections define concepts such as Instance, Database Engine, Configuration, ACL, Failover, Configuration, Backup and Restore in Scaleway Database.
A Database Engine is the software component that stores and retrieves your data from a database. For example, PostgreSQL is a database engine and it implements the SQL standard. Database Engines have versions. Different major versions represents changes that can break backward compatibility.
See "Engine Specifics" for supported engines and versions.
A Database Instance is composed of one or more computing nodes, depending on the chosen HA setting (1 leader for Standalone, 1 leader + 1 replica for High Availability). HA standby nodes are linked to a leader, using synchronous replication. Synchronous replication offers the ability to confirm that all changes intended by a transaction have been transferred and applied to one or more synchronous replica servers, thus providing durability to the data. You can also add up to 3 read replicas that can be used for some read-only workflows like BI, or for a read-only scaling of your application. Read replicas use asynchronous replication to replicate data from the leader.
A Database Instance has settings that let you tune the behavior of its Database Engine to better fit your needs. Available settings depend on the Database Engine and its version. Each instance setting entry has a default value that the user can override. The deletion of a setting entry will restore the setting to the entry's default value. Some of the defaults values can be different from the engine defaults, as we optimize these defaults for the Scaleway platform.
A Database Instance Endpoint is composed of an IP and a TCP port. These are the coordinates that are used to connect to the service. For now, there is only one endpoint, always pointing to the leader node of the Instance.
Endpoints IPs are public (No access is available using private IPs for now, but this might change in the future). This might raise security concerns to some, but here are a few points that must be remembered:
Please note that RDB architecture uses LoadBalancers to forward the traffic to the right node based on the node state (active/hot standby). LoadBalancers configuration is set to cut off inactive connections if no TCP traffic is sent within a 6-hour timeframe. We recommend using a connection pooling on application side in order to renew database connections regularly.
Access Control List Rules (ACL Rules) define which host and networks can connect to your Database Instance Endpoint. You can add, edit or delete rules from your ACLs. The initial setup of an instance allows full network access from anywhere (0.0.0.0/0).
Access control is handled directly at network-level by our load-balancers, so the Database does not need to do it itself, making the filtering more efficient, universal and relieving the Database from this task.
You can order a high availability database instance. With this configuration, we install the RDBMS (Relational Database Management System) on two dedicated compute nodes. We also make sure those instances are on two physically separate racks.
On a High Availability Database Instance, replication is synchronous. With this replication mode, write operations are routed to one database node. The other node (standby node) replicates these data. Write operations are acknowledged to the user only when they are applied on the master AND on the standby node. If a node goes down, data integrity is guaranteed. You can't be in the situation where the leader node has more data than the replica. This replication mode introduces a bit of overhead on the database system, but for us, data integrity is the most important.
High availability instances are monitored continuously and got an AutoHealing feature. If a database node goes down, the standby node will take write operations in less than 10 seconds letting you be able to continue your business as usual with read and write operations. During this phase, your instance switch to AUTOHEALING status (blue dot in the console). During this period, your High Availability Instance is working on the remaining database node for at most 700 seconds. After 700 seconds, if your database node is still considered as down, a new node is spawned and automatically configured as a standby database node. Once configured, your database instance get the READY status back. With this system, your database instance is unavailable only for a few seconds letting your app work as usual.
Logical Databases can be saved to and restored from our storage facilities, located in a different physical region than your Database. Each logical database inside of a database instance is backed up and therefore restorable separately.
A User is an account protected by a password, allowing access to your Database Instance.
There are two ways to manage user permssions: from the API/Console or using SQL. All supported engines have a rich permission model and as most users only need a part of it, so Scaleway Database provides a simplified and unified permission model through the API and the console to make things easier to manage and understand. Beside this, it is perfectly fine to use engine-specific ways of managing users and permissions, see each engine documentation to find mode about the corresponding SQL commands.
The following is only about the simplified workflow.
Each User has associated Permissions that give it access to zero or more Logical Databases. A User can have the "admin" flag, the behaviour is slightly different depending on the engine, see below in "Engine Specifics".
| Permission Level | Description |
|---|---|
none | no access to the database |
read | read-only access to the data |
read/write | read and write access to the data |
admin | read and write access to the data plus some more privileges |
| depending on the engine (see "Engine Specifics") |
Logs can contain useful information for debugging or to know more about the behaviour and activity of your database. They are available for download, but must first be gathered from the Database Nodes.
Memory, CPU and Disk usage are important metrics that you should keep an eye on. Graphs are available on the "Metrics" section in the console. You can also track these value from the API to inject them in you own system and raise custom alerts.
Scaleway's infrastructure is divided into regions and zones.
Scaleway Database is available in the following regions:
You can use cURL to send request to APIs, we will provide curl example for each step.
Go to the following link: https://console.scaleway.com/register Provide a valid e-mail address and a password.
Login to your Scaleway account and create a token from the credentials
page. A SECRET_KEY and an access_key will show on your screen, the
SECRET_KEY will be used as X-Auth-Token.
reference: https://www.scaleway.com/docs/generate-an-api-token/
In order to use Scaleway API you need to have curl installed. It is also a good idea to have jq, which will help you to read and parse JSON output. Make sure you have these two tools before you begin. Otherwise use your package manager to install them.
To call Scaleway API, you need an API key. If you don't have one yet, you can create it with the Scaleway console.
Using this token, get your project ID:
You can also find your project ID in the Scaleway web console.
This documentation shows how to create and connect to a PostgreSQL Database using Scaleway Database product.
Requirements:
You have to complete the "Setup" explained on upper sections, you are logged in scaleway and have your API Token and Project ID. We are going to create, connect to and delete a database.
Create your database instance
| Parameter | Do |
|---|---|
engine | database's engine version id |
name | database's name |
node_type | Choose your node type |
project_id | Your Scaleway project ID and X-Auth-Token are from your account |
is_ha_cluster | Enable or disable high availability for database instance |
disable_backup | Disable automated backup on your database instance |
tags | list of tags ["tag1", "tag2", ...] attached to a database instance |
user_name | Identifier of the first user of your database |
password | Password for the fist user |
Example:
Now you've created your database instance, you can open a connection between a database client and your database. You will need a PostgreSQL client such as psql
List instances
example:
Get the endpoint, port
Connect to your database with psql client
Congratulations! You have done your first steps on Database product !
In order to migrate your PostgreSQL instance to a Scaleway Database for PostgreSQL instance, you will need the following:
pg_dump command-line installedpsql command-line installedYou can export your data from your source instance to a local file by performing the following command:
If your source postgreSQL is in a docker container, the command will be:
In order to restore a database, you have to create the destination database and users of that database in your instance.
You can create your database and roles by using the web console, or by using the API. Ensure that your user permission
is correctly set to all.
Once you have created each destination database and its users, you can import your data from the <database>.sql file to your scaleway database instance by performing the following command:
Scaleway Database for PostgreSQL supports PostgreSQL versions 10, 11, 12, 13 and 14.
Some modules are available for installation, including the popular TimescaleDB and PostGIS. See FAQ for a list of allowed extensions.
Logical databases are created either using Console/API or SQL.
admin level have the CREATEROLE and CREATEDB privilegesSUPERUSER privilegesDatabase will be restored with the permissions they were backuped with. However, privileges will be ignored for deleted users, and if objects cannot be restored to the original owner, they will be owned by an internal system user.
Scaleway Database for MySQL supports MySQL 8.
mysql_native_password and caching_sha2_password authentication are supported.
Default is mysql_native_password for compatibility.admin level have access to all logical databases and can create new ones.Even if we manage your database storage space, you're responsible for its usage.
In the case you're reaching the space provisioned for your instance, the instance
will shift to the disk_full status, informing you that there is only 1GB left
on your database instance. This lets you either scale up the disk on the console
or leaves you enough room to do some cleanup.
In disk_full mode, the default transaction mode will shift to read only,
allowing your application to still access its data, while preventing new
data from being inserted.
In order to remove data from your instance, you must connect to it as usual,
but explicitly shift to a read write transaction, like this:
When enough space is available on the instance, the default transaction mode
will change back to read write shortly after, allowing normal operation again.
Failing to cleanup or forcing insertions while the database is in disk_full mode will
lead to downtime.
Save and restore backups of your database instance.
created_at_asc, created_at_desc, name_asc, name_desc, status_asc and status_desc. The default value is created_at_asc.unknown, creating, ready, restoring, deleting, error, exporting and locked. The default value is unknown.unknown, creating, ready, restoring, deleting, error, exporting and locked. The default value is unknown.unknown, creating, ready, restoring, deleting, error, exporting and locked. The default value is unknown.unknown, creating, ready, restoring, deleting, error, exporting and locked. The default value is unknown.unknown, creating, ready, restoring, deleting, error, exporting and locked. The default value is unknown.unknown, creating, ready, restoring, deleting, error, exporting and locked. The default value is unknown.Software that stores and retrieves data from a database. Each database engine has a name and versions.
A Database Instance is composed of one or more Nodes, depending of the is_ha_cluster setting. Autohealing is enabled by default for HA clusters. Database automated backup is enabled by default in order to prevent data loss.
created_at_asc, created_at_desc, name_asc, name_desc, region, status_asc and status_desc. The default value is created_at_asc.organization_id and project_id may be set.organization_id and project_id may be set.lssd and bssd. The default value is lssd.unknown, ready, provisioning, configuring, deleting, error, autohealing, locked, initializing, disk_full, backuping, snapshotting and restarting. The default value is unknown.unknown, ready, provisioning, configuring, deleting, error, autohealing, locked, initializing, disk_full, backuping, snapshotting and restarting. The default value is unknown.unknown, ready, provisioning, configuring, deleting, error, autohealing, locked, initializing, disk_full, backuping, snapshotting and restarting. The default value is unknown.unknown, ready, provisioning, configuring, deleting, error, autohealing, locked, initializing, disk_full, backuping, snapshotting and restarting. The default value is unknown.unknown, ready, provisioning, configuring, deleting, error, autohealing, locked, initializing, disk_full, backuping, snapshotting and restarting. The default value is unknown.Get database instance metrics.
Prepare your instance logs. Logs will be grouped on a minimum interval of a day.
unknown, ready, provisioning, configuring, deleting, error, autohealing, locked, initializing, disk_full, backuping, snapshotting and restarting. The default value is unknown.Upgrade your current node_type or enable high availability on your standalone database instance.
node_type, enable_ha, volume_size and volume_type may be set.node_type, enable_ha, volume_size and volume_type may be set.node_type, enable_ha, volume_size and volume_type may be set.unknown, ready, provisioning, configuring, deleting, error, autohealing, locked, initializing, disk_full, backuping, snapshotting and restarting. The default value is unknown.unknown, ready, creating and error. The default value is unknown.A read replica is a live copy of the main database instance only available for reading. Read replica allows you to scale your database instance for read-heavy database workloads. Read replicas can also be used for Business Intelligence workloads. Listing of read replicas is available in the instance response object. A read replica can have at most one direct access and one private network endpoint. Loadbalancer endpoint is not available on read replica even if this resource is displayed in the read replica response example. If you want to remove a read replica endpoint, you can use the instance delete endpoint API call. Instance Access Control List (ACL) also applies on read replica direct access endpoint. Don't forget to set it to improve the security of your read replica nodes. Be aware that there can be replication lags between the primary node and its read replica nodes. You can try to reduce this lag with some good practices:
You can only create a maximum of 3 read replicas for one instance.
unknown, provisioning, initializing, ready, deleting, error, locked and configuring. The default value is unknown.unknown, provisioning, initializing, ready, deleting, error, locked and configuring. The default value is unknown.unknown, provisioning, initializing, ready, deleting, error, locked and configuring. The default value is unknown.A read replica can have at most one direct access and one private network endpoint.
unknown, provisioning, initializing, ready, deleting, error, locked and configuring. The default value is unknown.When you resync a read replica, first it is reset, and then its data is resynchronized from the primary node. Your read replica will be unavailable during the resync process. The duration of this process is proportional to your Database Instance size. The configured endpoints will not change.
unknown, provisioning, initializing, ready, deleting, error, locked and configuring. The default value is unknown.Network Access Control List allows to control network in and out traffic by setting up ACL rules.
Add an additional ACL rule to a database instance.
Replace all the ACL rules of a database instance.
Instance Settings are tunables of Database Engines. Available settings depend on the database engine and its version.
Define some privileges to a user on a specific database.
user_name_asc, user_name_desc, database_name_asc and database_name_desc. The default value is user_name_asc.Manage users on your instance
Manage logical databases on your instance
name_asc, name_desc, size_asc and size_desc. The default value is name_asc.Node types powering your instance
Create, restore and manage block snapshot
unknown, creating, ready, restoring, deleting, error and locked. The default value is unknown.created_at_asc, created_at_desc, name_asc, name_desc, expires_at_asc and expires_at_desc. The default value is created_at_asc.unknown, creating, ready, restoring, deleting, error and locked. The default value is unknown.unknown, creating, ready, restoring, deleting, error and locked. The default value is unknown.unknown, creating, ready, restoring, deleting, error and locked. The default value is unknown.unknown, ready, provisioning, configuring, deleting, error, autohealing, locked, initializing, disk_full, backuping, snapshotting and restarting. The default value is unknown.Create and manage database instance endpoints
private_network, load_balancer and direct_access may be set.private_network, load_balancer and direct_access may be set.private_network, load_balancer and direct_access may be set.private_network, load_balancer and direct_access may be set.private_network, load_balancer and direct_access may be set.private_network, load_balancer and direct_access may be set.