Felipe Cruz – Docker https://www.docker.com Tue, 16 Jan 2024 20:21:58 +0000 en-US hourly 1 https://wordpress.org/?v=6.5.3 https://www.docker.com/wp-content/uploads/2024/02/cropped-docker-logo-favicon-32x32.png Felipe Cruz – Docker https://www.docker.com 32 32 Back Up and Share Docker Volumes with This Extension https://www.docker.com/blog/back-up-and-share-docker-volumes-with-this-extension/ Wed, 14 Sep 2022 14:30:00 +0000 https://www.docker.com/?p=37495

When you need to back up, restore, or migrate data from one Docker host to another, volumes are generally the best choice. You can stop containers using the volume, then back up the volume’s directory (such as /var/lib/docker/volumes/<volume-name>). Other alternatives, such as bind mounts, rely on the host machine’s filesystem having a specific directory structure available, for example /tmp/source on UNIX systems like Linux and macOS and C:/Users/John on Windows.

Normally, if you want to back up a data volume, you run a new container using the volume you want to back up, then execute the tar command to produce an archive of the volume content:

docker run --rm \
      -v "$VOLUME_NAME":/backup-volume \
      -v "$(pwd)":/backup \
      busybox \
      tar -zcvf /backup/my-backup.tar.gz /backup-volume

To restore a volume with an existing backup, you can run a new container that mounts the target volume and executes the tar command to decompress the archive into the target volume. 

A quick Google search returns a number of bash scripts that can help you back up volumes, like this one from Docker Captain Bret Fisher. With this script, you can get the job done with the simpler ./vackup export my-volume backup.tar.gz. While scripts like this are totally valid approaches, the Extensions team was wondering: what if we could integrate this tool into Docker Desktop to deliver a better developer experience? Interestingly enough, it all started as a simple demo just the day before going live on Bret’s streaming show!

Now you can back up volumes from Docker Desktop

You can now back up volumes with just a few clicks using the new Volumes Backup & Share extension. This extension is available in the Marketplace and works on macOS, Windows, and Linux. And you can check out the OSS code on GitHub to see how the extension was developed.

volumes backup share
How to back up a volume to a local file in your host

What can I do with the extension?

The extension allows you to:

  • Back up data that is persisted in a volume (for example, database data from Postgres or MySQL) into a compressed file.

  • Upload your backup to Docker Hub and share it with anyone.

  • Create a new volume from an existing backup or restore the state of an existing volume.

  • Transfer your local volumes to a different Docker host (through SSH).

  • Other basic volume operations like clone, empty, and delete a volume.

In the scenario below, John, Alex, and Emma are using Docker Desktop with the Volume Backup & Share extension. John is using the extension to share his volume (my-app-volume) with the rest of their teammates via Docker Hub. The volume is uploaded to Docker Hub as an image (john/my-app-volume:0.0.1) by using the “Export to Registry” option. His colleagues, Alex and Emma, will use the same extension to import the volume from Docker Hub into their own volumes by using the “Import from Registry” option.

share volume docker

Create different types of volume backups

When backing up a volume from the extension, you can select the type of backup:

  • A local file: creates a compressed file (gzip’ed tarball) in a desired directory of the host filesystem with the content of the volume.

  • A local image: saves the volume data into the /volume-data directory of an existing image filesystem. If you were to inspect the filesystem of this image, you will find the backup stored in /volume-data.

  • A new image: saves the volume data into the /volume-data directory of a newly created image.

  • A registry: pushes a local volume to any image registry, whether local (such as localhost:5000) or hosted like Docker Hub or GitHub Container Registry. This allows you to share a volume with your team with a couple clicks.

    >  As of today, the maximum volume size supported to push to Docker Hub by the extension is 10GB. This limit may be changed in future versions of the extension depending on feedback received from users.

Restore or import from a volume

Similarly to the different types of volume backups described above, you can import or restore a backup into a new or an existing volume.

You can also select whether you want to restore a volume from a local file, a local image, a new image, or from a registry.

Transfer a volume to another Docker host

You might also want to copy the content of a volume to another host where Docker is running (either Docker Engine or Docker Desktop), like an Ubuntu server or a Raspberry Pi.

From the extension, you can specify both the destination host the local volume copied to (for example, user@192.168.1.50) and the destination volume.

> SSH must be enabled and configured between the source and destination Docker hosts. Check to make sure you have the remote host SSH public key in your known_hosts file.

Below is an example of transferring a local volume from Docker Desktop to a Raspberry Pi.

transfer volume docker

Perform other operations

The extension provides other volume operations such as view, clone, empty, or delete.

How does it work behind the scenes?

In a nutshell, when a back up or restore operation is about to be carried out, the extension will stop all the containers attached to the specified volume to avoid data corruption, and then it will restart them once the operation is completed.

These operations happen in the background, which means you can carry out more of them in parallel, or leave the extension screen and navigate to other parts of Docker Desktop to continue with your work while the operations are running.

For instance, if you have a Postgres container that uses a volume to persist the database data (i.e. -v my-volume:/var/lib/postgresql/data), the extension will stop the Postgres container attached to the volume, generate a .tar.gz file with all the files that are inside the volume, then start the containers and put the file on the local directory that you have specified.

Note that for open files like databases, it’s usually better to use their preferred backup tool to create a backup file, but if you stored that file on a Docker volume, this could still be a way you get the Docker volume into an image or tarball for moving to remote storage for safekeeping.

What’s next?

We invite you to try out the extension and give us feedback here.

And if you haven’t tried Docker Extensions, we encourage you to explore the Extensions Marketplace and install some of them! You can also start developing your own Docker Extensions on all platforms: Windows, WSL2, macOS (both Intel and Apple Silicon), and Linux.

To learn more about the Extensions SDK, have a look at the official documentation. You’ll find tutorials, design guidelines, and everything else you need to build an extension.
Once your extension’s ready, you can submit it to the Extensions Marketplace here.

Related posts

]]> Build Your First Docker Extension https://www.docker.com/blog/build-your-first-docker-extension/ Thu, 12 May 2022 14:00:21 +0000 https://www.docker.com/?p=33572 At Docker, we’re passionate about building tools that make other developers’ lives easier. When it comes to building and running containers, Docker Desktop is the go-to product for developers. With Docker Desktop 4.8.0, you can now add your own functionality to Docker Desktop and impact the lives of millions.

This blog post walks you through the steps on how to develop, install, and preview a simple extension inside Docker Desktop. By the end of this tutorial, you will have an extension with a basic user interface, including a button that, when clicked, makes an HTTP request to the extension backend and displays the response in Docker Desktop.

Docker Extensions Beta Screenshot

You can develop extensions in any text editor and use any language for the frontend which can be translated to JavaScript (including JavaScript itself!). To deliver the best experience for your users, we recommend using TypeScript and React / Material UI for the frontend. If you want to create a backend service for your extension (e.g. a RESTful API), you can build one in a language of your choice (Go, Python, dotnet, etc). The backend runs inside a container, giving you total freedom over your language and framework choices.

What a time to be a developer! Let’s get started building your first extension!

Prerequisites

To complete this tutorial, you’ll need:

Create the extension files

The quickest way to get started with your Docker Extension is to use the init command of the Docker Extensions CLI:

docker extension init my-extension

You’ll be prompted with a few questions about the extension that you want to build. These questions help the CLI to generate a set of boilerplate files for you in the my-extension directory.

create docker extension cli

The extension directory structure

The init command populates the given directory with all the files needed to define a Docker Extension:

  • The ui directory, which contains a React app that makes an HTTP request to the backend service and displays the response.
  • The vm directory, which contains a Go backend service (HTTP server) that listens on a socket and exposes the /hello endpoint that returns a JSON payload.
  • Some other files like metadata.json, which contains information about the extension.
extensions drop down menu

Build and install the extension

The init command creates a Makefile at the root of the extension directory. This makes it easier to build the extension for multiple platforms, install it locally, and push it to Docker Hub.

A Docker Extension is a Docker image, so the first step is to build the extension image.

A multi-stage Dockerfile is provided that consists of three stages: a first stage to build the backend service, a second for the React app, and a final stage where we copy the backend binary and the React assets. Finally, the CMD instruction starts the backend service, using a socket to listen for requests from the React app.

You can use docker build or the predefined make build-extension command to build the extension.

extensions code felipe

The previous command generates a Docker image named after the Docker Hub repository you provided to the init command.  For this guide, we’re using john/my-extension. Feel free to use the same inputs for the purpose of this practice exercise. You won’t be able to push to that repository, but all the local development steps will still work.

To install the extension in Docker Desktop, run:

docker extension install john/my-extension

You should see the following output:

Installing new extension "john/my-extension:latest"
Installing service in Desktop VM...
Setting additional compose attributes
VM service started
Installing Desktop extension UI for tab "My-Extension"...
Extension UI tab "My-Extension" added.
Extension "my-extension" installed successfully

If you open Docker Desktop, on the left-hand menu you’ll see a new tab with the name My-Extension. Click on it to navigate to your extension and see the React app. When you click on the Call backend button, you’ll see {"Message":"hello"}displayed.

Docker Extensions Beta Screenshot

You can also check that the extension has been installed using the following CLI command:

docker extension ls

Output:

ID                  PROVIDER            VERSION             UI                    VM                  HOST
john/my-extension   John                latest              1 tab(My-Extension)   Running(1)          -

Make a change in the extension code

In order to make a change in the extension code and see the change, you need to re-build your image and then update your extension. We provide some shortcuts to these commands so that you don’t need to worry about:

make build-extension
make update-extension

Behind the scenes these commands are doing just:

docker build -t john/my-extension
docker extension update john/my-extension

The extension is reinstalled in Docker Desktop with your code changes and you can test it again.

Enable hot reloading

If you’re developing the Frontend side of the extension there’s no need to rebuild your image after every change, you can use the hot reload that’s built-in. Within this example, we are using React, with a simple create-react-app project, that exposes the Frontend on http://localhost:3000.

cd ui
npm start
docker extension dev ui-source john/my-extension http://localhost:3000

Close and reopen the Docker Desktop dashboard and go to your extension, all the changes to the frontend code are immediately visible after saving any code changes.

Once you’re done updating and testing your extension, you can remove the ui-source override by running:

docker extension dev reset john/my-extension

Open Chrome Dev Tools

In order to open the Chrome Dev Tools for your extension when clicking on the extension tab, run:

docker extension dev debug john/my-extension

Each subsequent click on the extension tab will also open Chrome Dev Tools. To stop this behavior, run:

docker extension dev reset john/my-extension

Change the backend service code

If you need to make changes in the backend service, you’ll also need to build the extension again:

make build-extension
make update-extension

Because in this example your extension has a backend service running as a container, the update command removes the backend container and starts the new one.

Styling your extension

Docker Desktop’s UI is written in React and Material-UI. We strongly recommend adopting this combination in your extension as well. This brings the benefit of using our Docker Material UI Theme to easily replicate Docker Desktop’s look & feel.

Check out our design guidelines.

Preview the extension in the Marketplace

When you build and install your unpublished extension, you can preview it in the Marketplace Installed tab. This allows you to see how the extension description, screenshots and links are displayed on the details page of the extension before users  install the extension.
You can provide all that information using LABELs in the Dockerfile:

LABEL org.opencontainers.image.title="my-extension" \
   org.opencontainers.image.description="My first extension" \
   org.opencontainers.image.vendor="John" \
   com.docker.desktop.extension.api.version=">= 0.2.3" \
   com.docker.extension.screenshots="[{\"alt\": \"docker build\", \"url\": \"https://www.docker.com/wp-content/uploads/2021/09/Docker-build.png\"}]" \
   com.docker.extension.detailed-description="My detailed description &#x1f433;" \
   com.docker.extension.publisher-url="" \
   com.docker.extension.additional-urls="[{\"title\":\"Documentation\",\"url\":\"https:\/\/foo.bar\/docs\"},{\"title\":\"Support\",\"url\":\"https:\/\/foo.bar\/support\"}}]]"
my extensions screenshot

What’s next?

If you haven’t tried Docker Extensions, we highly encourage you to explore the Extensions Marketplace and install some of them! You can start developing Docker Extensions on all platforms: Windows, WSL2, Mac Intel and Apple Silicon, and Linux.

To learn more about the Extensions SDK, have a look at the official documentation. You’ll find tutorials, design guidelines, and everything you need to build Docker Extensions.

Do you want to submit your extension to the Extensions Marketplace? Click here.

Do you want to submit feedback? Click here

Related posts

]]>