Docker Hub – Docker https://www.docker.com Thu, 16 May 2024 20:13:09 +0000 en-US hourly 1 https://wordpress.org/?v=6.5.3 https://www.docker.com/wp-content/uploads/2024/02/cropped-docker-logo-favicon-32x32.png Docker Hub – Docker https://www.docker.com 32 32 Docker and JFrog Partner to Further Secure Docker Hub and Remove Millions of Imageless Repos with Malicious Links https://www.docker.com/blog/docker-jfrog-partner-to-further-secure-docker-hub/ Tue, 30 Apr 2024 14:00:55 +0000 https://www.docker.com/?p=54468 Like any large platform on the internet (such as GitHub, YouTube, GCP, AWS, Azure, and Reddit), Docker Hub, known for its functionality and collaborative environment, can become a target for large-scale malware and spam campaigns. Today, security researchers at JFrog announced that they identified millions of spam repositories on Docker Hub without images that have malicious links embedded in the repository descriptions/metadata. To be clear, no malicious container images were discovered by JFrog. Rather, these were pages buried in the web interface of Docker Hub that a user would have to discover and click on to be at any risk. We thank our partner JFrog for this report, and Docker has deleted all reported repositories. Docker also has a security@docker.com mailbox, which is monitored by the Security team. All malicious repositories are removed once validated.

2400x1260 dockerjfrog

The JFrog report highlights methods employed by bad actors, such as using fake URL shorteners and Google’s open redirect vulnerabilities to mask their malicious intent. These attacks are not simple to detect — many are not malware but simple links, for example, and wouldn’t be detectable except by humans or flagged as malicious by security tools. 

JFrog identified millions of “imageless” repositories on Docker Hub. These repositories, devoid of actual Docker images, serve merely as fronts for distributing malware or phishing attacks. Approximately 3 million repositories were found to contain no substantive content, just misleading documentation intended to lure users to harmful websites. The investment in maintaining Hub is enormous on many fronts.

These repositories are not high-traffic repositories and would not be highlighted within Hub. The below repository is an example highlighted in JFRog’s blog. Since there is not an image in the repository, there will not be any pulls.

docker jfrog security screenshot 1

An image would be displayed below with a corresponding tag. These repositories are empty.

docker jfrog security screenshot 2

Conclusion

Docker is committed to security and has made substantial investments this past year, demonstrating our commitment to our customers. We have recently completed our SOC 2 Type 2 audit and ISO 27001 certification review, and we are waiting on certification. Both SOC 2 and ISO 27001 demonstrate Docker’s commitment to Customer Trust and securing our products. 

We urge all Docker users to use trusted content. Docker Hub users should remain vigilant, verify the credibility of repositories before use, and report any suspicious activities. If you have discovered a security vulnerability in one of Docker’s products or services, we encourage you to report it responsibly to security@docker.com. Read our Vulnerability Disclosure Policy to learn more.

Docker is committed to collaborating with security experts like JFrog and the community to ensure that Docker Hub remains a safe and robust platform for developers around the globe. 

]]>
Docker Partners with NVIDIA to Support Building and Running AI/ML Applications https://www.docker.com/blog/docker-nvidia-support-building-running-ai-ml-apps/ Mon, 18 Mar 2024 22:06:12 +0000 https://www.docker.com/?p=53020 The domain of GenAI and LLMs has been democratized and tasks that were once purely in the domain of AI/ML developers must now be reasoned with by regular application developers into everyday products and business logic. This is leading to new products and services across banking, security, healthcare, and more with generative text, images, and videos. Moreover, GenAI’s potential economic impact is substantial, with estimates it could add trillions of dollars annually to the global economy. 

Docker offers an ideal way for developers to build, test, run, and deploy the NVIDIA AI Enterprise software platform — an end-to-end, cloud-native software platform that brings generative AI within reach for every business. The platform is available to use in Docker containers, deployable as microservices. This enables teams to focus on cutting-edge AI applications where performance isn’t just a goal — it’s a necessity.

This week, at the NVIDIA GTC global AI conference, the latest release of NVIDIA AI Enterprise was announced, providing businesses with the tools and frameworks necessary to build and deploy custom generative AI models with NVIDIA AI foundation models, the NVIDIA NeMo framework, and the just-announced NVIDIA NIM inference microservices, which deliver enhanced performance and efficient runtime. 

This blog post summarizes some of the Docker resources available to customers today.

docker nvidia 2400x1260 1

Docker Hub

Docker Hub is the world’s largest repository for container images with an extensive collection of AI/ML development-focused container images, including leading frameworks and tools such as PyTorch, TensorFlow, Langchain, Hugging Face, and Ollama. With more than 100 million pull requests for AI/ML-related images, Docker Hub’s significance to the developer community is self-evident. It not only simplifies the development of AI/ML applications but also democratizes innovation, making AI technologies accessible to developers across the globe.

NVIDIA’s Docker Hub library offers a suite of container images that harness the power of accelerated computing, supplementing NVIDIA’s API catalog. Docker Hub’s vast audience — which includes approximately 27 million monthly active IPs, showcasing an impressive 47% year-over-year growth — can use these container images to enhance AI performance. 

Docker Hub’s extensive reach, underscored by an astounding 26 billion monthly image pulls, suggests immense potential for continued growth and innovation.

Docker Desktop with NVIDIA AI Workbench

Docker Desktop on Windows and Mac helps deliver NVIDIA AI Workbench developers a smooth experience on local and remote machines. 

NVIDIA AI Workbench is an easy-to-use toolkit that allows developers to create, test, and customize AI and machine learning models on their PC or workstation and scale them to the data center or public cloud. It simplifies interactive development workflows while automating technical tasks that halt beginners and derail experts. AI Workbench makes workstation setup and configuration fast and easy. Example projects are also included to help developers get started even faster with their own data and use cases.   

Docker engineering teams are collaborating with NVIDIA to improve the user experience with NVIDIA GPU-accelerated platforms through recent improvements to the AI Workbench installation on WSL2.

Check out how NVIDIA AI Workbench can be used locally to tune a generative image model to produce more accurate prompted results:

In a near-term update, AI Workbench will use the Container Device Interface (CDI) to govern local and remote GPU-enabled environments. CDI is a CNCF-sponsored project led by NVIDIA and Intel, which exposes NVIDIA GPUs inside of containers to support complex device configurations and CUDA compatibility checks. This simplifies how research, simulation, GenAI, and ML applications utilize local and cloud-native GPU resources.  

With Docker Desktop 4.29 (which includes Moby 25), developers can configure CDI support in the daemon and then easily make all NVIDIA GPUs available in a running container by using the –device option via support for CDI devices.

docker run --device nvidia.com/gpu=all <image> <command>

LLM-powered apps with Docker GenAI Stack

The Docker GenAI Stack lets teams easily integrate NVIDIA accelerated computing into their AI workflows. This stack, designed for seamless component integration, can be set up on a developer’s laptop using Docker Desktop for Windows. It helps deliver the power of NVIDIA GPUs and NVIDIA NIM to accelerate LLM inference, providing tangible improvements in application performance. Developers can experiment and modify five pre-packaged applications to leverage the stack’s capabilities.

Accelerate AI/ML development with Docker Desktop

Docker Desktop facilitates an accelerated machine learning development environment on a developer’s laptop. By tapping NVIDIA GPU support for containers, developers can leverage tools distributed via Docker Hub, such as PyTorch and TensorFlow, to see significant speed improvements in their projects, underscoring the efficiency gains possible with NVIDIA technology on Docker.

Securing the software supply chain

Securing the software supply chain is a crucial aspect of continuously developing ML applications that can run reliably and securely in production. Building with verified, trusted content from Docker Hub and staying on top of security issues through actionable insights from Docker Scout is key to improving security posture across the software supply chain. By following these best practices, customers can minimize the risk of security issues hitting production, improving the overall reliability and integrity of applications running in production. This comprehensive approach not only accelerates the development of ML applications built with the Docker GenAI Stack but also allows for more secure images when building on images sourced from Hub that interface with LLMs, such as LangChain. Ultimately, this provides developers with the confidence that their applications are built on a secure and reliable foundation.

With exploding interest in AI from a huge range of developers, we are excited to work with NVIDIA to build tooling that helps accelerate building AI applications. The ecosystem around Docker and NVIDIA has been building strong foundations for many years and this is enabling a new community of enterprise AI/ML developers to explore and build GPU accelerated applications.”

Justin Cormack, Chief Technology Officer, Docker

Enterprise applications like NVIDIA AI Workbench can benefit enormously from the streamlining that Docker Desktop provides on local systems. Our work with the Docker team will help improve the AI Workbench user experience for managing GPUs on Windows.”

Tyler Whitehouse, Principal Product Manager, NVIDIA

Learn more 

By leveraging Docker Desktop and Docker Hub with NVIDIA technologies, developers are equipped to harness the revolutionary power of AI, grow their skills, and seize opportunities to deliver innovative applications that push the boundaries of what’s possible. Check out NVIDIA’s Docker Hub library  and NVIDIA AI Enterprise to get started with your own AI solutions.

]]>
NVIDIA AI Workbench | Fine Tuning Generative AI nonadult
Azure Container Registry and Docker Hub: Connecting the Dots with Seamless Authentication and Artifact Cache https://www.docker.com/blog/azure-container-registry-and-docker-hub-connecting-the-dots-with-seamless-authentication-and-artifact-cache/ Thu, 29 Feb 2024 14:48:05 +0000 https://www.docker.com/?p=52583 By leveraging the wide array of public images available on Docker Hub, developers can accelerate development workflows, enhance productivity, and, ultimately, ship scalable applications that run like clockwork. When building with public content, acknowledging the potential operational risks associated with using that content without proper authentication is crucial. 

In this post, we will describe best practices for mitigating these risks and ensuring the security and reliability of your containers.

Black padlock on light blue digital background

Import public content locally

There are several advantages to importing public content locally. Doing so improves the availability and reliability of your public content pipeline and protects you from failed CI builds. By importing your public content, you can easily validate, verify, and deploy images to help run your business more reliably.

For more information on this best practice, check out the Open Container Initiative’s guide on Consuming Public Content.

Configure Artifact Cache to consume public content

Another best practice is to configure Artifact Cache to consume public content. Azure Container Registry’s (ACR) Artifact Cache feature allows you to cache your container artifacts in your own Azure Container Registry, even for private networks. This approach limits the impact of rate limits and dramatically increases pull reliability when combined with geo-replicated ACR, allowing you to pull artifacts from the region closest to your Azure resource. 

Additionally, ACR offers various security features, such as private networks, firewall configuration, service principals, and more, which can help you secure your container workloads. For complete information on using public content with ACR Artifact Cache, refer to the Artifact Cache technical documentation.

Authenticate pulls with public registries

We recommend authenticating your pull requests to Docker Hub using subscription credentials. Docker Hub offers developers the ability to authenticate when building with public library content. Authenticated users also have access to pull content directly from private repositories. For more information, visit the Docker subscriptions page. Microsoft Artifact Cache also supports authenticating with other public registries, providing an additional layer of security for your container workloads.

Following these best practices when using public content from Docker Hub can help mitigate security and reliability risks in your development and operational cycles. By importing public content locally, configuring Artifact Cache, and setting up preferred authentication methods, you can ensure your container workloads are secure and reliable.

Learn more about securing containers

Additional resources for improving container security for Microsoft and Docker customers

]]>
Using Authenticated Logins for Docker Hub in Google Cloud https://www.docker.com/blog/authenticated-logins-docker-hub-in-google-cloud/ Tue, 19 Dec 2023 15:11:35 +0000 https://www.docker.com/?p=49877 The rise of open source software has led to more collaborative development, but it’s not without challenges. While public container images offer convenience and access to a vast library of prebuilt components, their lack of control and potential vulnerabilities can introduce security and reliability risks into your CI/CD pipeline.

This blog post delves into best practices that your teams can implement to mitigate these risks and maintain a secure and reliable software delivery process. By following these guidelines, you can leverage the benefits of open source software while safeguarding your development workflow.

Using Authenticated Logins for DockerHub in GoogleCloud 2400x1260 1

1. Store local copies of public containers

To minimize risks and improve security and reliability, consider storing local copies of public container images whenever feasible. The Open Containers Initiative offers guidelines on consuming public content, which you can access for further information.

2. Use authentication when accessing Docker Hub

For secure and reliable CI/CD pipelines, authenticating with Docker Hub instead of using anonymous access is recommended. Anonymous access exposes you to security vulnerabilities and increases the risk of hitting rate limits, hindering your pipeline’s performance.

The specific authentication method depends on your CI/CD infrastructure and Google Cloud services used. Fortunately, several options are available to ensure secure and efficient interactions with Docker Hub.

3. Use Artifact Registry remote repositories 

Instead of directly referencing Docker Hub repositories in your build processes, opt for Artifact Registry remote repositories for secure and efficient access. This approach leverages Docker Hub access tokens, minimizing the risk of vulnerabilities and facilitating a seamless workflow.

Detailed instructions on configuring this setup can be found in the following Artifact Registry documentation: Configure remote repository authentication to Docker Hub.

authenticated dockerhub login

4. Use Google Cloud Build to interact with Docker images 

Google Cloud Build offers robust authentication mechanisms to pull Docker Hub images seamlessly within your build steps. These mechanisms are essential if your container images rely on external dependencies hosted on Docker Hub. By implementing these features, you can ensure secure and reliable access to the necessary resources while streamlining your CI/CD pipeline.

Implementing the best practices outlined above offers significant benefits for your CI/CD pipelines. You’ll achieve a stronger security posture and reduced reliability risks, ensuring smooth and efficient software delivery. Additionally, establishing robust authentication controls for your development environments prevents potential roadblocks that could arise later in production. As a result, you can be confident that your processes comply with or surpass corporate security standards, further solidifying your development foundation.

Learn more

Visit the following product pages to learn more about the features that assist you in implementing these steps.

]]>
Maximizing Software Development’s ROI: Forrester’s TEI Study of Docker Business https://www.docker.com/blog/forresters-tei-study-of-docker-business/ Mon, 18 Dec 2023 17:01:30 +0000 https://www.docker.com/?p=49872 Docker’s commitment to empowering developers and organizations is evident in its ongoing investment in the Docker Business subscription, which includes Docker Desktop, Docker Hub, and Docker Scout. Through collaborative efforts with a vibrant user community and customers, Docker has pioneered best practices and innovations that significantly streamline application development workflows. 

Today, Docker Business — Docker’s solution that supports organizations of every size in optimizing DevOps, CI/CD, debugging, and IT processes — marks a significant step in enhancing enterprise development efficiency. The recent Forrester Total Economic Impact™ (TEI) study commissioned by Docker underscores for us the measurable benefits experienced by Docker Business users, including accelerated development agility, reduced time-to-market, and substantial cost savings.

Docker TEI Forrester Study Blog 2400x1260 v2

Maximizing resource efficiency with Docker 

Docker Business transforms the developer experience by simplifying workflows across multiple development phases. While its prowess lies in optimizing DevOps, CI/CD, and IT processes, Docker’s solution does this by consistently supporting a higher quality and more intuitive management experience for seamlessly packaging, distributing, and executing applications across diverse computing environments.

This way of offering a combined developer interface and toolset makes creating containerized applications easier. This approach also reduces complications linked to separate solutions and old-style virtual machines (VMs), making data centers work more efficiently.

Enhanced security and rapid deployment

Security remains a pivotal focus for Docker Business, employing robust measures like isolation and encryption to safeguard applications and data. The streamlined development cycles enabled by Docker Business expedite application deployment and testing, fostering a culture of innovation and agility within enterprises.

Key insights from the Forrester TEI™ Study

Forrester conducted comprehensive interviews with representatives from top global technology manufacturers, consolidated and referred to as a composite organization, uncovering Docker Business’s capacity to tackle issues associated with slow legacy systems and costly VM dependencies. The study also shows compelling statistics highlighting Docker Business’s impact on the composite organization:

  • 6% increase in application developer productivity
  • Improved DevOps engineer-to-developer ratio from 1:20 to 1:60
  • 3x reduction in servers due to increased VM density
  • 3 months faster time-to-market for revenue-generating applications

Embrace Docker Business for transformational outcomes

The transformative potential of Docker Business is evident in its effective resolution of legacy system challenges and dependency on traditional VMs with a secure and flexible development platform built to ensure enterprises, teams, and developers’ success. Docker Business opens the door to remarkable benefits for organizations by enhancing developer velocity, accelerating development agility, reducing time-to-market, and delivering substantial cost savings to the business.

DevOps and IT Productivity: $10.1M Application developer productivity: $18.8M Reduced data center capacity requirement for legacy apps: $3.9M Reduced data center capacity requirement for new apps: $69.9M Net operating profit due to improved time to market of new apps: $17.4M

Download the full Forrester Total Economic Impact™ (TEI) study to learn more about how Docker Business with Docker Desktop, Docker Hub, and Docker Scout fosters a positive total economic impact.

Learn more

]]>
Docker Hub Registry IPv6 Support Now Generally Available https://www.docker.com/blog/docker-hub-registry-ipv6-support-now-generally-available/ Wed, 23 Aug 2023 14:29:31 +0000 https://www.docker.com/?p=43023 As the world becomes increasingly interconnected, it’s essential for the internet to support the growing number of devices and users. That’s where IPv6 comes in.

Graphic showing IPv6 text in blue circle with networking lines on light blue background

What is IPv6, and what does it have to do with Docker? 

IPv6 is the latest version of the Internet Protocol, the system that enables devices to communicate with each other over the internet. It’s designed to address the limitations of the previous version, IPv4, which is running out of available addresses. 

As Docker supports more customers, this means we need to support different use cases, like IPv6-only networks. Today, we are pleased to announce the general availability of IPv6 support for the Docker Hub Registry, Docker Docs, and Docker Scout endpoints. 

Why are we adopting IPv6? 

We have heard from the community that you need IPv6 support for Docker software as a service (SaaS) endpoints to work efficiently and effectively. In the past, IPv6-only networks required extra tooling to interact with some of Docker’s SaaS resources. This is no longer the case. Now you can get rid of your NAT64 gateway and docker pull.

What does this mean for my workflows? 

This is my favorite part… nothing! 🥳 During our beta testing of IPv6, we introduced new endpoints for accessing the Docker Hub Registry. Those were only for the beta testing and are no longer needed. Now, if you are on an IPv6-only network, dual-stack network, or an IPv4-only network, these commands will work.

To begin, log in to the Docker Hub:

docker login

Then pull whatever image you need:

docker pull alpine

How will Docker Hub download rate limits work?

If you use authentication when pulling container images from the Docker Hub Registry, nothing changes. Our servers will properly attach rate limit data to the authenticated user ID in the HTTP request. 

If you do not authenticate your docker pull commands by running docker login first, then we’ll need to rate limit the request based on the IP address. For IPv4 addresses, this is done on a per-IP basis. 

For IPv6 addresses, this becomes a harder problem because IPv6 has a much larger IP address range available to customers. Therefore, to accommodate the larger IP address range, we will rate limit against the first 64 bits in the IPv6 address. You can see an example of what our servers use as the source by looking at the docker-ratelimit-source header returned in the following HTTP response:

$ curl https://registry-1.docker.io/v2/ratelimitpreview/test/manifests/latest -I -XGET -6
HTTP/1.1 401 Unauthorized
content-type: application/json
docker-distribution-api-version: registry/2.0
www-authenticate: Bearer realm="https://auth.docker.io/token",service="registry.docker.io",scope="repository:ratelimitpreview/test:pull"
date: Wed, 28 June 2023 01:06:44 GMT
content-length: 164
strict-transport-security: max-age=31536000
docker-ratelimit-source: 2601:245:c100:a71::

How can I verify that IPv6 is being used? 

While browsing the Docker Docs or pulling a Docker container image, you can use network monitoring software like tcpdump to monitor the traffic.

Let’s say that you want to verify the network you use for pulling a container image from the Docker Hub Registry.

First, in your favorite terminal, start a tcpdump capture. This command will produce log records of all of the network connections between your local machine and the Docker Hub servers:

sudo tcpdump host registry-1.docker.io -vv

In another terminal window, pull a container image from Docker Hub:

docker pull registry-1.docker.io/library/alpine:latest

You should see output that looks like this:

&#x1f680; sudo tcpdump host registry-1.docker.io -vv
tcpdump: data link type PKTAP
tcpdump: listening on pktap, link-type PKTAP (Apple DLT_PKTAP), snapshot length 524288 bytes
15:42:16.740577 IP6 (flowlabel 0xa0800, hlim 64, next-header TCP (6) payload length: 44) 2601:245:c100:a71:8454:86d0:52f1:d46f.62630 > 2600:1f18:2148:bc02:cfd8:db68:ea1f:277c.https: Flags [S], cksum 0xb80b (correct), seq 2539670618, win 65535, options [mss 1440,nop,wscale 6,nop,nop,TS val 4154959809 ecr 0,sackOK,eol], length 0
15:42:16.774831 IP6 (class 0x20, hlim 229, next-header TCP (6) payload length: 40) 2600:1f18:2148:bc02:cfd8:db68:ea1f:277c.https > 2601:245:c100:a71:8454:86d0:52f1:d46f.62630: Flags [S.], cksum 0x6b60 (correct), seq 4264170311, ack 2539670619, win 26847, options [mss 1440,sackOK,TS val 2058512533 ecr 4154959809,nop,wscale 12], length 0
…

When you look at the second column of tab-delimited data, it will say IP6 to denote IPv6 being used. Additionally, the IP addresses you see in the output are in IPv6 format instead of IPv4 format. The quick way to tell is if the IP address has a colon (:) in it, then it is IPv6. If the IP address only has periods (.), then it is IPv4. 🎉

The future

We are excited to be able to provide full dual-stack network capabilities to Docker Hub Registry, Docker Docs, and Docker Scout endpoints. We believe that dual-stack capabilities offer an important performance and reliability benefit to our customers. We intend to provide dual-stack network support for new endpoints as part of our commitment to delivering the best possible experience for our users. 

If you have the ability to control your local network, turn on IPv6 and see Docker Hub Registry, Docker Docs, and Docker Scout endpoints continue to work. If you have access to an IPv6-only network, try docker pull or take a look at our docs pages — they will all continue to work as they did before. 

We look forward to hearing feedback from our community through our hub-feedback GitHub issue tracker.

Learn more

]]>
Memgraph Docker Extension: Empowering Real-Time Analytics with High Performance https://www.docker.com/blog/memgraph-docker-extension-empowering-real-time-analytics-with-high-performance/ Fri, 04 Aug 2023 13:29:15 +0000 https://www.docker.com/?p=44167 Memgraph is an open source, in-memory graph database designed with real-time analytics in mind. Providing a high-performance solution, Memgraph caters to developers and data scientists who require immediate, actionable insights from complex, interconnected data.

What sets Memgraph apart is its high-speed data processing ability, delivering performance that makes it significantly faster than other graph databases. This, however, is not achieved at the expense of data integrity or reliability. Memgraph is committed to providing accurate and dependable insights as fast as you need them.

Built entirely on a C++ codebase, Memgraph leverages in-memory storage to handle complex real-time use cases effectively. Support for ACID transactions guarantees data consistency, while the Cypher query language offers a robust toolset for data structuring, manipulation, and exploration. 

Graph databases have a broad spectrum of applications. In domains as varied as cybersecurity, credit card fraud detection, energy management, and network optimization, Memgraph can efficiently analyze and traverse complex network structures and relationships within data. This analytical prowess facilitates real-time, in-depth revelations across a broad spectrum of industries and areas of study. 

In this article, we’ll show how using Memgraph as a Docker Extension offers a powerful and efficient way to leverage real-time analytics from a graph database. 

Graphic showing Docker and Memgraph logos on light blue background.

Architecture of Memgraph

The high-speed performance of Memgraph can be attributed to its unique architecture (Figure 1). Centered around graph models, the database represents data as nodes (entities) and edges (relationships), enabling efficient management of deeply interconnected data essential for a range of modern applications.

In terms of transactions, Memgraph upholds the highest standard. It uses the standardized Cypher query language over the Bolt protocol, facilitating efficient data structuring, manipulation, and exploration.

Illustration of Memgraph components, including mgconsole, Kafka, C API, MAGE, etc.
Figure 1: Components of Memgraph’s architecture.

The key components of Memgraph’s architecture are:

  • In-memory storage: Memgraph stores data in RAM for low-latency access, ensuring high-speed data retrieval and modifications. This is critical for applications that require real-time insights.
  • Transaction processing: Memgraph supports ACID (Atomicity, Consistency, Isolation, Durability) transactions, which means it guarantees that all database transactions are processed reliably and in a way that ensures data integrity, including when failures occur.
  • Query engine: Memgraph uses Cypher, a popular graph query language that’s declarative and expressive, allowing for complex data relationships to be easily retrieved and updated.
  • Storage engine: While Memgraph primarily operates in memory, it also provides a storage engine that takes care of data durability by persisting data on disk. This ensures that data won’t be lost even if the system crashes or restarts.
  • High availability and replication: Memgraph’s replication architecture can automatically replicate data across multiple machines, and it supports replication to provide high availability and fault tolerance.
  • Streaming and integration: Memgraph can connect with various data streams and integrate with different types of data sources, making it a versatile choice for applications that need to process and analyze real-time data.

To provide users with the utmost flexibility and control, Memgraph comprises several key components, each playing a distinct role in delivering seamless performance and user experience:

  • MemgraphDB — MemgraphDB is the heart of the Memgraph system. It deals with all concurrency problems, consistency issues, and scaling, both in terms of data and algorithm complexity. Using the Cypher query language, MemgraphDB allows users to query data and run algorithms. It also supports both push and pull operations, which means you can query data and run algorithms and get notified when something changes in the data.
  • Mgconsole — mgconsole is a command-line interface (CLI) used to interact with Memgraph from any terminal or operating system. 
  • Memgraph Lab — Memgraph Lab is a visual user interface for running queries and visualizing graph data. It provides a more interactive experience, enabling users to apply different graph styles, import predefined datasets, and run example queries. It makes data analysis and visualization more intuitive and user-friendly.
  • MAGE (Memgraph Advanced Graph Extensions) — MAGE is an open source library of graph algorithms and custom Cypher procedures. It enables high-performance processing of demanding graph algorithms on streaming data. With MAGE, users can run a variety of algorithms, from PageRank or community detection to advanced machine learning techniques using graph embeddings. Moreover, MAGE does not limit users to a specific programming language.

Based on those four components, Memgraph offers four different Docker images:

With more than 10K downloads from Docker Hub, Memgraph Platform is the most popular Memgraph Docker image, so the team decided to base the Memgraph Docker extension on it. Instructions are available in the documentation if you want to use any of the other images. Let’s look at how to install Memgraph Docker Extension.

Why run Memgraph as a Docker Extension?

Running Memgraph as a Docker Extension offers a streamlined experience to users who are already familiar with Docker Desktop, simplifying the deployment and management of the graph database. Docker provides an ideal environment to bundle, ship, and run Memgraph in a lightweight, isolated setup. This encapsulation not only promotes consistent performance across different systems but also simplifies the setup process.

Moreover, Docker Desktop is the only prerequisite to run Memgraph as an extension. This means that once you have Docker installed, you can easily set up and start using Memgraph, eliminating the need for additional software installations or complex configuration steps.

Getting started

Working with Memgraph as a Docker Extension begins with opening the Docker Desktop (Figure 2). Here are the steps to follow:

  1. Choose Extensions in the left sidebar.
  2. Switch to the Browse tab.
  3. In the Filters drop-down, select the Database category.
  4. Find Memgraph and then select Install
Screenshot of Extensions Marketplace showing Docker Extensions.
Figure 2: Installing Memgraph Docker Extension.

That’s it! Once the installation is finished, select Connect now (Figure 3).

Screenshot of Docker Desktop showing orange Connect Now button you can use to connect to Memgraph.
Figure 3: Connecting to Memgraph database using Memgraph Lab.

What you see now is Memgraph Lab, a visual user interface designed for running queries and visualizing graph data. With a range of pre-prepared datasets, Memgraph Lab provides an ideal starting point for exploring Memgraph, gaining proficiency in Cypher querying, and effectively visualizing query results.  

Importing the Pandora Papers datasets

For the purposes of this article, we will import the Pandora Papers dataset. To import the dataset, choose Datasets in the Memgraph Lab sidebar and then Load Dataset (Figure 4).

 Screenshot of Docker Desktop showing Pandora Papers as featured dataset.
Figure 4: Importing the Pandora Papers dataset.

Once the dataset is loaded, select Explore Query Collection to access a selection of predefined queries (Figure 5).

Screenshot of Docker Desktop showing orange button to Explore Query Collection.
Figure 5: Exploring the Pandora Papers dataset query collection.

Choose one of the queries and select Run Query (Figure 6).

Screenshot of Docker Desktop showing query in the Cypher editor.
Figure 6: Running the Cypher query.

And voilà! Welcome to the world of graphs. You now have the results of your query (Figure 7). Now that you’ve run your first query, feel free to explore other queries in the Query Collection, import a new dataset, or start adding your own data to the database.

Screenshot of Docker Desktop showing the query result as a graph.
Figure 7: Displaying the query result as a graph.

Conclusion

Memgraph, as a Docker Extension, offers an accessible, powerful, and efficient solution for anyone seeking to leverage real-time analytics from a graph database. Its unique architecture, coupled with a streamlined user interface and a high-speed query engine, allows developers and data scientists to extract immediate, actionable insights from complex, interconnected data.

Moreover, with the integration of Docker, the setup and use of Memgraph become remarkably straightforward, further expanding its appeal to both experienced and novice users alike. The best part is the variety of predefined datasets and queries provided by the Memgraph team, which serve as excellent starting points for users new to the platform.

Whether you’re diving into the world of graph databases for the first time or are an experienced data professional, Memgraph’s Docker Extension offers an intuitive and efficient solution. So, go ahead and install it on Docker Desktop and start exploring the intriguing world of graph databases today. If you have any questions about Memgraph, feel free to join Memgraph’s vibrant community on Discord.

Learn more

]]>
Protecting Secrets with Docker https://www.docker.com/blog/protecting-secrets-with-docker/ Thu, 03 Aug 2023 14:23:12 +0000 https://www.docker.com/?p=44628 Modern software is interconnected. When you develop an application, it has to communicate with other services — on your infrastructure, cloud infrastructure services, or third-party applications. Of course, you don’t want just anyone to masquerade as you, so you use secrets like SSH keys or API tokens to make the communication secure. But having these secrets means you have to keep them secret.

Unfortunately, sometimes the secrets escape. When this happens, it can allow bad actors to maliciously use the secrets or post them on the “dark web” for others to use. They can insert vulnerabilities into your code. They can impersonate you or deny legitimate users access to resources. And, if the secret is for something billable (like public cloud infrastructure), they can cost you a lot of money. No matter what other costs you face, the public relations impact can cause your users to lose trust in you.

In this article, we’ll cover a few ways that Docker can help keep secrets from leaking.

Before you read on: If your secrets have been exposed, the first step is to immediately invalidate them and check for compromises.

banner protecting secrets with docker

Controlling access with Docker Hub

The principle of least privilege is a powerful part of your security posture. If someone doesn’t need access to your Docker Hub images, they shouldn’t have access. Docker Hub provides private repositories so that you can keep your images to yourself. Docker Personal subscribers can create one private repository, while Docker Pro, Docker Team, and Docker Business subscriptions offer unlimited private repositories.

Keep in mind that even with private repositories, Docker Hub is not for storing account secrets. Private repositories are a layer in your defense-in-depth model.

Of course, sometimes you want to selectively share your images. Docker Pro, Docker Team, and Docker Business subscribers can add collaborators — accounts that can push or pull images in a private repository. Docker Pro subscribers can add one collaborator to a repository. Docker Teams and Docker Business subscribers can add up to the organization size. This means you can share images with the people who need them — and no one else.

Keeping secrets out

What’s better than protecting the secrets on your Docker image? Not having them in the image in the first place! While there are cases where you need to store a secret in order to make the proper connections, many cases of secret leakage involve secrets that were added accidentally.

The best way to avoid accidentally adding secrets is to use a secret manager, such as AWS Secrets Manager, HashiCorp Vault, or 1Password, which has some CLI options. If you have to keep the secrets in a local environment, you can prevent files from accidentally winding up on your image by adding them to the .dockerignore file. For example, if you’re worried about accidentally adding SSH keys to your image, you can include: *id_rsa*

This approach works well for secrets in files with predictable names. If you’re always storing your cloud credentials in a file called cloud_key.txt, then you’re well-covered. But you won’t catch cloud_credentials.txt.

You can add another layer of security with secret scanners. Tools like Aqua Security Trivy, Anchore, and JFrog Xray search your files for things that look like secrets. If you run the scanner before pushing your image, then you can catch the secret before it escapes. Many secrets scanners can be tied into a Git commit hook as well to prevent secrets from being included in your code.

Conclusion

Keeping your secrets secret is an ongoing process but worth the effort. Like everything in cybersecurity, there’s no one magic solution, but Docker provides features that you can use to help prevent leaking secrets.

To get the most from private repositories and collaborators, check out our subscription offerings. We’re considering adding secret scanning to Docker Scout. If you’d like to see this capability, upvote the issue on our public roadmap.

Learn more

]]>
Why Are There More Than 100 Million Pull Requests for AI/ML Images on Docker Hub? https://www.docker.com/blog/100-million-pull-requests-for-ai-ml-images-docker-hub/ Thu, 20 Jul 2023 15:59:40 +0000 https://www.docker.com/?p=44111 A quick look at pull requests of well-known AI/ML-related images on Docker Hub shows more than 100 million pull requests. What is driving this level of demand in the AI/ML space? The same things that drive developers to use Docker for any project: accelerating development, streamlining collaboration, and ensuring consistency within projects. 

In this article, we’ll look more closely at how Docker provides a powerful tool for AI/ML development.

Graphic showing "AI/ML Images" text in web search box along with the words "hot topic" and a small fire emoji.

As we interact with more development teams who use Docker as part of their AI/ML efforts, we are learning about new and exciting use cases and hearing first-hand how using Docker has helped simplify the process of sharing AI/ML solutions with their teams and other AI/ML practitioners.

Why is Docker the deployment choice for millions of developers when working with AI/ML?

AI/ML development involves managing complex dependencies, libraries, and configurations, which can be challenging and time-consuming. Although these complexities are not limited to AI/ML development, with AI/ML, they can be more taxing on developers. Docker, however, has been helping developers address such issues for 10 years now.

Consistency across environments

Docker allows you to create a containerized environment that includes all the dependencies required for your AI/ML project, including libraries, tools, and frameworks. This environment can be easily shared and replicated across different machines and operating systems, ensuring consistency and reproducibility. Docker images can also be version-controlled and shared via container registries such as Docker Hub, thus enabling seamless collaboration and continuous integration and delivery.

Scalability

Docker provides a lightweight and efficient way to scale AI/ML applications. With Docker, you can run multiple containers on the same machine or across different machines in a cluster, enabling horizontal scaling. This approach can help you handle large datasets, run multiple experiments in parallel, and increase the overall performance of your applications.

Portability

Docker provides portability, allowing you to run your AI/ML applications on any platform that supports Docker, including local machines, cloud-based infrastructures, and edge devices. Docker images can be built once and deployed anywhere, eliminating compatibility issues and reducing the need for complex configurations. This can help you streamline the deployment process and focus on the development of your models.

Reproducibility

Docker enables reproducibility by providing a way to package the entire AI/ML application and its dependencies into a container. This container can be easily shared and replicated, ensuring that experiments are reproducible, regardless of the environment they are run in. Docker provides a way to specify the exact versions of dependencies and configurations needed to reproduce results, which can help validate experiments and ensure reliability and repeatability.

Easy collaboration

Docker makes it easy to collaborate on AI/ML projects with team members or colleagues. Docker images or containers can be easily shared and distributed, ensuring that everyone has access to the same environment and dependencies. This collaboration can help streamline the development process and reduce the time and effort required to set up development environments.

Conclusion

Docker provides a powerful tool for AI/ML development, providing consistency, scalability, portability, reproducibility, and collaboration. By using Docker to package and distribute AI/ML applications and their dependencies, developers can simplify the development process and focus on building and improving their models. 

Check out the Accelerated AI/ML Development page to learn more about how Docker fits into the AI/ML development process.

If you have an interesting use case or story about Docker in your AI/ML workflow, we would love to hear from you and maybe even share your story.

Learn more

]]>
160% Year-over-Year Growth in Pulls of Red Hat’s Universal Base Image on Docker Hub https://www.docker.com/blog/blog-red-hat-universal-base-image-hub-pulls-grow/ Thu, 25 May 2023 14:00:00 +0000 https://www.docker.com/?p=42852 Red Hat’s Universal Base Image eliminates “works on my machine” headaches for developers.

It’s Red Hat Summit week, and we wanted to use this as an opportunity to highlight several aspects of Docker’s partnership with Red Hat. In this post, we highlight Docker Hub and Red Hat’s Universal Base Images (UBI). Also check out our new post on simplifying Kubernetes development with Docker Desktop + Red Hat OpenShift.

Docker Hub is the world’s largest public registry of artifacts for developers, providing the fundamental building blocks — web servers, runtimes, databases, and more — for any application. It offers more than 15 million images for containers, serverless functions, and Wasm with support for multiple operating systems (Linux, Windows) and architectures (x86, ARM). Altogether, these 15 million images are pulled more than 16 billion times per month by over 20 million IPs.

Red Hat logo with verified publisher button and text that shows 160% increase in pulls, with a docker logo at the bottom

While the majority of the 15 million images are community images, a subset are trusted content, both open source and commercial, curated and actively maintained by Docker and upstream open source communities and Docker’s ISV partners.

Docker and Red Hat have been partners since 2013, and Red Hat started distributing Linux images through Docker Hub in 2014. To help developers reduce the “works on my machine” finger-pointing and ensure consistency between development and production environments, in 2019 Red Hat launched Universal Base Image (UBI). Based on Red Hat Enterprise Linux (RHEL), UBI provides the same reliability, security, and performance as RHEL. Furthermore, to meet the different use cases of developers and ISVs, UBI comes in four sizes — standard, minimal, multi-service, and micro — and offers channels so that additional packages can be added as needed.

Given Docker’s reach in the developer community and the breadth and depth of developer content on Docker Hub, Docker and Red Hat agreed that distributing Red Hat UBI on Docker Hub made a lot of sense. Thus, in May 2021 Red Hat became a Docker Verified Publisher (DVP) and launched Red Hat UBIs on Docker Hub. As a DVP, Red Hat’s UBIs are easier for developers to discover, and it gives developers an extra level of assurance that the images they’re using are accessible, safe, and maintained.

The results? Tens of 1000s of developers are pulling Red Hat UBI millions of times every month. Furthermore, the pulls of Red Hat Universal Base image have grown 2.6X times in the last 12 months alone. Such growth points to value Docker and Red Hat together bring to the developer community.

… and we’re not finished! Having provided Red Hat UBIs directly to developers, now Docker and Red Hat are working together with Docker’s ISV partners and open source communities to bring the value of UBI to those software stacks as well. Stay tuned for more!

Learn More

]]>