Remote image processing

Perform server-side processing of a user's image via an editing extension.

Remote image processing lets editing extensions offload image processing tasks to a server, instead of handling them via the browser. This makes it possible to manipulate user's images with CPU and GPU-intensive tasks, such as those that involve machine learning.

This tutorial explains how to:

  • Set up a REST API to handle remote image processing.

  • Trigger remote image processing via the JavaScript client.

  • Process the user’s image on a server.

The following pages contain the reference documentation for remote image processing:

Step 1: Set up a Base URL

In the Developer Portal, remote image processing extensions have a Base URL field. This indicates that the extension must provide the URL of a server that can receive and respond to HTTP requests from Canva.

To set up a Base URL, refer to the following guides:

You can use any programming language, framework, or architecture to handle requests from Canva. The examples in this documentation use Express.js.

Step 2: Enable remote image processing

  1. Navigate to an app via the Developer Portal.

  2. From the Extensions page, expand the Editing panel.

  3. Select Extension type > Remote image processing.

  4. Select Input file types > JPG/PNG.

  5. In the Base URL field, enter the URL of the server.

Step 3: Call the remoteProcess method

The JavaScript client for editing extensions has a remoteProcess method that initiates the remote processing of the user's image. You can call this method at various points in the lifecycle of the extension, such as when a user selects a preset or adjusts the value of a rich control.

The following snippet demonstrates how to call the remoteProcess method when a user clicks a button:

const { imageHelpers } = window.canva;
const canva = window.canva.init();
const state = {
canvas: null,
};
canva.onReady(async (opts) => {
const image = await imageHelpers.fromElement(opts.element);
state.canvas = await imageHelpers.toCanvas(image);
document.body.appendChild(state.canvas);
renderControls();
});
canva.onControlsEvent(async (event) => {
const result = await canva.remoteProcess();
console.log(result);
});
function renderControls() {
const controls = [
canva.create('button', {
id: 'startRemoteImageProcessing',
label: 'Start remote image processing...',
}),
];
canva.updateControlPanel(controls);
}

If you await on the remoteProcess method from the onReady callback, the onReady callback will likely time out.

Step 4: Handle image processing requests

When an extension calls the remoteProcess method, Canva sends a POST request to the following endpoint:

<base_url>/editing/image/process

The body of the request includes an imageUrl property that contains the URL of the user's image. You can use this URL to download and process the user's image on the server.

The following snippet uses jimp to download the image, invert its colors, and output the processed file to a directory named "public":

app.post('/editing/image/process', async (request, response) => {
// Download the user's image
jimp.read(request.body.imageUrl).then((image) => {
// Create a file path for the processed image
const imageUrl = request.body.imageUrl.split('?')[0];
const fileName = new Date().getTime() + path.extname(imageUrl);
const filePath = path.join(__dirname, 'public', fileName);
// Process the user's image
image.invert().write(filePath);
});
});

When the server has processed the image, it can respond with the URL of the image. The following snippet uses the url module to construct a URL for the image:

app.post('/editing/image/process', async (request, response) => {
// Download the user's image
jimp.read(request.body.imageUrl).then((image) => {
// Create a file path for the processed image
const imageUrl = request.body.imageUrl.split('?')[0];
const fileName = new Date().getTime() + path.extname(imageUrl);
const filePath = path.join(__dirname, 'public', fileName);
// Process the user's image
image.invert().write(filePath);
// Give Canva the processed image
response.send({
type: 'SUCCESS',
resource: {
url: url.format({
protocol: 'https',
host: request.get('host'),
pathname: fileName,
}),
width: image.bitmap.width,
height: image.bitmap.height,
type: 'JPG',
},
});
});
});

Image URLs must be HTTPS-enabled and accept cross-origin requests.

(Optional) Step 5: Handle polling requests

It's not always practical for a server to immediately process the user's image, so to avoid timeout errors for longer-running tasks, Canva supports polling.

An extension must support polling if it can't process a user's image within 15 seconds (or within 30 seconds for apps released in China).

To enable polling, provide an id in response to the /editing/image/process request. The id must uniquely identify the image processing task. In a production environment, the id may correspond to the ID of a background job. In this tutorial, the id is the name of the file.

The following snippet starts processing the user's image when the request is received but responds to the request before the processing completes:

app.post('/editing/image/process', async (request, response) => {
// Create a file path for the processed image
const imageUrl = request.body.imageUrl.split('?')[0];
const fileName = new Date().getTime() + path.extname(imageUrl);
const filePath = path.join(__dirname, 'public', fileName);
// Download the user's image
jimp.read(request.body.imageUrl).then((image) => {
// Process the user's image
image.invert().write(filePath);
});
// Give Canva an ID for the image
response.send({
type: 'SUCCESS',
id: fileName,
});
});

Next, create an endpoint with a path of /editing/image/process/get:

app.post('/editing/image/process/get', async (request, response) => {
// code goes here
});

Once Canva receives an id, it begins to send POST requests to the /editing/image/process/get endpoint. These requests continue at regular intervals until either of the following conditions are met:

  • The endpoint responds with the URL of the processed image.

  • The polling times out. (The current timeout length is 60 seconds.)

The previously defined id appears in the body of each request. This lets the server check if the image processing task has finished. If it has, the server can respond with the URL of the processed image.

The following snippet responds with the URL of the processed image if the processed image file exists:

app.post('/editing/image/process/get', async (request, response) => {
const fileName = request.body.id;
const filePath = path.join(__dirname, 'public', fileName);
jimp
.read(filePath)
.then((image) => {
response.send({
type: 'SUCCESS',
resource: {
url: url.format({
protocol: 'https',
host: request.get('host'),
pathname: fileName,
}),
width: image.bitmap.width,
height: image.bitmap.height,
type: getContentType(filePath),
},
});
})
.catch(() => {
response.send({
type: 'SUCCESS',
id: fileName,
});
});
});
/**
Calculate the value of the resource.type property based on
the extension of the image file.
*/
function getContentType(filePath) {
const extension = path.extname(filePath).toLowerCase();
if (extension === '.jpg' || extension === '.jpeg') {
return 'JPG';
}
if (extension === '.png') {
return 'PNG';
}
throw new Error(`Can't find content type for file path: ${filePath}`);
}

If the image file doesn't exist, the endpoint responds with the id provided in the initial request. This tells Canva to continue sending requests to /editing/image/process/get.

Step 6: Render the processed image

When the /editing/image/process or /editing/image/process/get endpoint responds with the URL of a processed image, Canva creates two versions of the user's image: a full-sized version and a preview-sized version. The remoteProcess method then returns an object that contains the URLs of each version:

canva.onControlsEvent(async (event) => {
const result = await canva.remoteProcess();
console.log(result.previewImage.url);
console.log(result.fullImage.url);
});

You can use the image helpers to render the processed image:

canva.onControlsEvent(async (event) => {
// Remotely process an image
const result = await canva.remoteProcess();
// Download the processed image
const image = await imageHelpers.fromUrl(result.previewImage.url);
// Render the image
const context = state.canvas.getContext('2d');
const img = await imageHelpers.toImageElement(image);
context.drawImage(img, 0, 0, state.canvas.width, state.canvas.height);
});

Step 7: Show a loading spinner

While an extension remotely processes an image, use the toggleSpinner method to show and hide a loading spinner:

canva.onControlsEvent(async (event) => {
// Show the loading spinner
canva.toggleSpinner('preview', true);
// Remotely process the user's image
const result = await canva.remoteProcess();
// Download the processed image
const image = await imageHelpers.fromUrl(result.previewImage.url);
// Render the image
const context = state.canvas.getContext('2d');
const img = await imageHelpers.toImageElement(image);
context.drawImage(img, 0, 0, state.canvas.width, state.canvas.height);
// Hide the loading spinner
canva.toggleSpinner('preview', false);
});

This reduces the perceived duration of the image processing.

(Optional) Step 8: Pass data to (and from) the server

Sometimes, extensions need to pass arbitrary data from the frontend to the backend (or from the backend to the frontend). For example, if an extension uses rich controls to provide users with configurable options, the backend needs access to that data. This section explains how to pass data in either direction.

Passing data to the backend

The remoteProcess method accepts an optional string via the opts.settings parameter. A string is a seemingly limited data type, but you can use the JSON.stringify method to pass more complex data:

const result = await remoteProcess({
settings: JSON.stringify({
name: 'David',
age: 30,
location: 'Australia',
}),
});

You can then access this data in the body of the /editing/image/process request, via the settings property:

app.post('/editing/image/process', async (request, response) => {
const json = JSON.parse(request.body.settings);
console.log(json.name);
});

Passing data to the frontend

The /editing/image/process and /editing/image/process/get endpoints can include a resource.metadata property in their responses:

response.send({
type: 'SUCCESS',
resource: {
url: 'https://picsum.photos/200/300',
width: 200,
height: 300,
type: 'JPG',
metadata: JSON.stringify({
name: 'David',
age: 30,
location: 'Australia',
}),
},
});

You can then access this data on the frontend via the metadata property:

const result = await remoteProcess();
const json = JSON.parse(result.metadata);
console.log(json.name);

(Optional) Step 9: Pass blobs to (and from) the server

Sometimes, it can be useful to pass blobs (e.g. JSON files, PNG files, BIN files) between the frontend and the backend. This section explains how to pass blobs in either direction.

Passing blobs to the backend

The remoteProcess method accepts a map of Blob objects via the opts.blobs parameter:

const jsonBlobExample = new Blob(
[JSON.stringify({ hello: 'world' }, null, 2)],
{ type: 'application/json' },
);
const result = await remoteProcess({
blobs: {
myJsonBlob: jsonBlobExample,
},
});

The keys in this object are IDs for the blobs, while the values are the blobs themselves.

Canva uploads these blobs to a server and makes them available to download via URLs. The server can access these URLs in the body of the /editing/image/process request:

app.post('/editing/image/process', async (request, response) => {
request.body.blobs.forEach((blob) => {
console.log(blob.id);
console.log(blob.type);
console.log(blob.url);
});
});

Passing blobs to the frontend

The /editing/image/process and /editing/image/process/get endpoints can include a resource.blobs property in their responses:

response.send({
type: 'SUCCESS',
resource: {
url: 'https://picsum.photos/200/300',
width: 200,
height: 300,
type: 'JPG',
blobs: [
{
id: 'myImage',
type: 'JPG',
url: 'https://picsum.photos/200/300',
},
],
},
});

Each item in the response.blobs array must be an object with the following properties:

  • id

  • type

  • url

Canva uploads each blob to a server. Your extension can access the URLs of these blobs on the frontend via the blobUrls property:

const result = await remoteProcess();
console.log(result.blobUrls);

Example

client.js

const { imageHelpers } = window.canva;
const canva = window.canva.init();
const state = {
canvas: null,
};
canva.onReady(async (opts) => {
const image = await imageHelpers.fromElement(opts.element);
state.canvas = await imageHelpers.toCanvas(image);
document.body.appendChild(state.canvas);
renderControls();
});
canva.onControlsEvent(async (event) => {
// Show the loading spinner
canva.toggleSpinner('preview', true);
// Remotely process an image
const result = await canva.remoteProcess();
// Download the processed image
const image = await imageHelpers.fromUrl(result.previewImage.url);
// Render the image
const context = state.canvas.getContext('2d');
const img = await imageHelpers.toImageElement(image);
context.drawImage(img, 0, 0, state.canvas.width, state.canvas.height);
// Hide the loading spinner
canva.toggleSpinner('preview', false);
});
function renderControls() {
const controls = [
canva.create('button', {
id: 'startRemoteImageProcessing',
label: 'Start remote image processing...',
}),
];
canva.updateControlPanel(controls);
}

server.js

const express = require('express');
const jimp = require('jimp');
const path = require('path');
const url = require('url');
const app = express();
app.use(express.json());
app.use(express.static('public'));
app.post('/editing/image/process', async (request, response) => {
// When Canva tests the signature of this endpoint, the provided
// "imageUrl" is "https://test.url", which causes jimp to throw an
// error. This conditional prevents jimp from throwing the error.
if (request.body.imageUrl.endsWith('test.url')) {
response.send({
type: 'ERROR',
errorCode: 'INTERNAL_ERROR',
});
return;
}
// Create a file path for the processed image
const imageUrl = request.body.imageUrl.split('?')[0];
const fileName = new Date().getTime() + path.extname(imageUrl);
const filePath = path.join(__dirname, 'public', fileName);
// Download the user's image
jimp.read(request.body.imageUrl).then((image) => {
// Process the user's image
image.invert().write(filePath);
});
// Give Canva an ID for the image
response.send({
type: 'SUCCESS',
id: fileName,
});
});
app.post('/editing/image/process/get', async (request, response) => {
// Get the file path for the processed image
const fileName = request.body.id;
const filePath = path.join(__dirname, 'public', fileName);
jimp
.read(filePath)
.then((image) => {
// The processed file exists, so give Canva the processed image
response.send({
type: 'SUCCESS',
resource: {
url: url.format({
protocol: 'https',
host: request.get('host'),
pathname: fileName,
}),
width: image.bitmap.width,
height: image.bitmap.height,
type: getContentType(filePath),
},
});
})
.catch(() => {
// The processed file doesn't exist, so continue polling
response.send({
type: 'SUCCESS',
id: fileName,
});
});
});
function getContentType(filePath) {
const extension = path.extname(filePath).toLowerCase();
if (extension === '.jpg' || extension === '.jpeg') {
return 'JPG';
}
if (extension === '.png') {
return 'PNG';
}
throw new Error(`Can't find content type for file path: ${filePath}`);
}
app.listen(process.env.PORT || 3000);