MnQ API (beta)v1alpha1

Scaleway Messaging is a Message broker offering notifications, queues, fifos and streams. This service is designed to enable you to easily make your applications communicate with each others and port your Microservice Applications to the cloud. It includes all major features of a modern message broker (like message persistence). When reaching general availability, it will enable you to choose between several protocols for your broker (named "namespaces" below).

Update: The SQS protocol has been added !

In the current product stage, two protocol options are available:

1. NATS: all features typically offered by a NATS v2 message broker are supported (pub/sub, request/response, streams, ...).

2. AWS SQS/SNS: it allows the use the AWS SDKs for using Queues and Topics. For now, SNS only supports subscriptions with the following protocols:

   • sqs
   • http
   • https (self-signed certificate are not supported)
   • lambda (used to call a Scaleway Serverless function)

• A single region is supported, your namespace will be located in Paris, France (fr-par).
• Only a subset of the SQS/SNS protocol is supported, see the pages SQS API Support and SNS API Support for more information.
• The product is in beta stage, we do not guarantee performances nor stability, be careful when using it as you may encounter data loss.

The following tutorial will guide you to create your first MnQ Namespace and use it to make two applications communicate thought a managed Pub/Sub messaging system.

In this tutorial we will use the NATS protocol to publish and receive messages, and to call the MnQ API we will use the well-known cURL HTTP client.

Alternatively, you can use the software Postman to make the following requests from a nice user interface. here is the tutorial to use Postman.

1. First of all, ensure you have installed the following tools:

• curl
• jq: a json query command line tool (should be provided by your package manager or can downloaded from here)
• nats cli: The NATS command line interface can be downloaded from here

2. Set your Scaleway token's secret key in the environment variable SCW_SECRET_KEY and the id of the Scaleway project you want to use into the variable SCW_DEFAULT_PROJECT_ID (you can use your Scaleway organisation id as it is your default project id).

3. Create your first Namespace.

4. Create the set of credentials. It will be needed in the next step to connect to your Namespace using the NATS client.

5. Open two terminals, one to subscribe to a subject, and another one to publish a message on this subject. On the first one execute the following command:

On the second one execute the following command:

The following links might be useful if you are new to NATS or want to go further.

• What is NATS
• Installing & using nats cli
• As JetStream is enabled with quotas on our servers, here is the offical documentation.
• Official NATS github: This is where you can find official client library for many languages (golang, python, rust, ...) and many other coding resources.

Also, you can interact directly with the NATS community on their Slack Channel.

Both SQS and SNS share the same namespace. On the API, the corresponding namespace protocol parameter value is sqs_sns.

1. First of all, ensure you have installed the following tools:

• curl
• jq: a json query command line tool (should be provided by your package manager or can downloaded from here)
• aws cli and it's endpoint plugin: You can refer to Scaleway's Object Storage documentation here

2. Set your Scaleway token's secret key in the environment variable SCW_SECRET_KEY and the id of the Scaleway project you want to use into the variable SCW_DEFAULT_PROJECT_ID (you can use your Scaleway organisation id as it is your default project id).

3. Create a Namespace with protocol sqs_sns.

4. Create the set of credentials. It will be needed in the next step to configure your Namespace in the aws cli configuration.

5. Configure the AWS Cli

• Either update your default profile or create a new one with the following configuration in your ~/.aws/config file:

  Where the endpoint_url value was returned by your namespace creation:

• create the corresponding credentials in ~/.aws/credentials:

  where <sqs_sns_credentials.access_key> and <sqs_sns_credentials.secret_key> are the values returned by the credentials API call:

• you can now define the AWS_PROFILE environment variable to use the newly created profile:

The following tutorial will guide you to create a namespace and to communicate using the SQS protocol.

1. Please follow the Prerequisites to getting started with SQS and/or SNS section, to create the required common resources to use SQS and SNS protocol.

2. Create a Queue

3. List existing Queues

4. Send messages to a Queue

5. Receive messages

6. Delete messages

   Once the messages have been processed on your consumer side (typically by a worker), the message needs to be deleted otherwise it will be re-queued.

7. Delete the queue itself

The following tutorial will guide you to create a namespace and to communicate using the SNS protocol.

1. Please follow the Prerequisites to getting started with SQS and/or SNS section, to create the required common resources to use SQS and SNS protocol.

2. Create a Topic

3. List existing Topics

4. Prepare a target for SNS messages and subscribe the SNS topic to the target (procedure depends on the chosen protocol)

   • for SQS:

     1. Create a Queue (the queue must be in the same namespace than the SNS topic)

     2. Get the Queue ARN

     3. Configure a subscription to push each new message sent on this topic to the queue

   • for HTTP/HTTPS:

     1. Get the public endpoint of the HTTP server you want to forward your messages to.

     2. Configure a subscription to push each new message sent on the topic to the HTTP server

     3. To complete the subscription process, you need to confirm the subscription using a link sent to your HTTP server.

        The HTTP server should have received an HTTP request with a body in json matching the following format:

     4. Confirm subscription

   • for lambda (Scaleway Serverless function):

     1. Create the function following the Quick Start Guide of Scaleway functions.

     2. Get the function endpoint from the Scaleway Console under "Functions" -> "[YOUR-FUNCTION-NAMESPACE]" -> "[YOUR-FUNCTION-NAME]" -> "Function Settings" tab -> "Function Endpoint"

        Warning: Only the main generated Endpoint of the function will work, not the aliases. The endpoint should match the following format:

     3. Configure a subscription to push each new message sent on this topic to the function

5. List subscriptions

6. Publish a message on the topic

7. Read the message received on the chosen target

   • for SQS:

     1. Read the message from this queue

     2. Delete the message

        As seen in the getting started with SQS section, the message needs to be deleted.

   • for HTTP/HTTPS, you should have received the message on your server

   • for lambda, your function should have been called with the message as argument

8. Delete the subscription

9. Delete the remaining resources of this tutorial

   • for SQS, delete the queue

   • for lambda, delete the function using the Scaleway Console

10. Delete the topic

SQS queues and SNS topics are compatible with the Amazon AWS SQS/SNS protocols. Please check out the official documentations for more information:

• Amazon Simple Queue Service Documentation.
• Amazon Simple Notification Service Documentation

The differences between the Scaleway service and the Amazon service are:

• The SQS/SNS protocol support is not complete, see the SQS/SNS APIs compatibility matrixes below

• The credentials use a simplified permissions system: each credential created by the CreateCredential API call gives permissions to the whole namespace. There are 3 permissions:

  • can_publish : Allows to send messages to a Queue/Topic.
  • can_receive : Allows to receive and acknowledge (delete) messages and to subscribe to a Topic.
  • can_manage : Allows all other actions (Creating, Listing, Updating, Deleting Queues/Topics).

  The permissions required by each action are listed in the following sections.

• The SNS/SQS credentials are issued by the Scaleway Messaging product, these are not your Scaleway API keys.

To ensure everything will go as smoothly as possible for this beta, we set the following limits:

• The maximum size of a message is set to 256 kB.
• The maximum storage is set to 100 MB per namespace accross all its streams.

We are available on Scaleway Community Slack on the #messaging-queuing-beta channel.

Create a set of credentials for a specific namespace.

Update a set of credentials.