Unlocking Even More Creative AI Workflows: Introducing RunwayML Integration in Griptape Nodes

Griptape Nodes
Written By
Ian Massingham
June 19, 2025

We’re excited to announce another powerful new integration in Griptape Nodes supporting integration with RunwayML’s suite of AI models. This addition empowers users to incorporate image-to-video generation directly into their visual AI workflows, with no code required. Whether you’re an artist, creative professional, developer, or business user, this integration opens up new possibilities for creative content creation.

What is the RunwayML Node Library?

The new RunwayML Node Library is an open-source Python library that brings RunwayML’s cutting-edge AI models into the Griptape Nodes ecosystem. With this integration, you can now add RunwayML-powered steps to your workflows, combining them with other data sources, APIs, and automation logic, all within Griptape Nodes. The benefits include:

  • Creative AI at Your Fingertips: Access RunwayML’s image and video models without writing code.
  • Composable Workflows: Combine RunwayML nodes with other Griptape nodes for end-to-end automation.
  • Rapid Innovation: Prototype and deploy AI-powered content workflows in minutes.

What’s Included?  

The RunwayML Node Library repository includes three nodes that let you interact with RunwayML’s API. Here’s a summary of what these nodes enable:

  • RunwayML Image to Video: Generates a video from a reference image and a text prompt using the RunwayML API.
  • RunwayML Text to Image: Generates an image from a text prompt and an optional list of reference images using the RunwayML API.
  • RunwayML Create Reference Image: Creates a ReferenceImageArtifact for use as an entry in the reference_images list in an instance of a RunwayML Text to Image node. Reference images can be used to combine different objects or characters into an image, and for image editing and style transfer use-cases.

How It Works in Practice  

Imagine you’re building a workflow to automate video content creation for your marketing team. With the RunwayML Node Library, you can:

  1. Create a Text Prompt: Start with a user input. We recommend experimenting with using agents to enhance your prompt. 
  2. Add optional reference images: Have a product, person, or character you want to appear in your image? Use the Reference Image feature within RunwayML together with the Create Reference Image node to create reference images to use in your creation.
  3. Generate an Image: Pass the prompt to the RunwayML Text to Image node, together with your optional reference images.
  4. Generate video from your image: Pass the image you created in the prior step to the Image to Video node, together with a text prompt describing the shot. In a few seconds, your video will be created and available for you to view.

How about an example?

In the example workflow shown below, we use two agents with OpenAI’s gpt-4.1-mini model to generate two enhanced prompts for image generation. 

The prompts that we use for the agents are:

create a detailed and descriptive image generation prompt about "a rusty blue pickup truck made out of ice cream" racing around a racetrack. perspective should be isometric. No more than 100 words

and

create a detailed and descriptive image generation prompt about "a yellow formula 1 car made out of ice cream" racing around a racetrack. perspective should be isometric. No more than 100 words

We connect the output of these agents to two RunwayML Text-to-Image nodes and the output of those nodes to two RunwayML Create Reference Image nodes where we associate the two generated images with the tags RaceCar and PickupTruck. The two Create Reference Image nodes are then connected to another Text-to-Image node with the the prompt:

The cars from @RaceCar and @PickupTruck driving down a busy city street at night.

You can see that we reference the reference images with @RaceCar and @PickupTruck in the prompt, and that the character consistency from doing so is excellent. Accurate use of reference images is an area where RunwayML is particularly effective. Finally, we pass the output image from this node to Image to Video node with the additional text prompt:

the vehicles drive down the street quickly. the camera pans to follow

How to Get Started  

To add RunwayML capabilities to your Griptape Nodes workflows:

  1. Clone the repository from GitHub and follow the instructions in the README on GitHub to install the library.
  2. Add your RunwayML API key with the name RUNWAYML_API_SECRET  in the API Keys & Secrets section of the Configuration Editor in Griptape Nodes
  3. Add RunwayML nodes to your workflow and configure them by entering prompts, selecting models, and aspect ratios for the outputs
  4. Run your workflow, monitor results, and refine as needed.

For detailed instructions, visit the griptape-nodes-library-runwayml GitHub repository.

The RunwayML integration brings further creative tools to Griptape Nodes, making it easier than ever to build, deploy, and share AI-powered creative workflows. Whether you’re automating content generation, enhancing media assets, or experimenting with the latest in AI, this new capability puts powerful tools at your fingertips.

Ready to explore what’s possible? Visit the Griptape Nodes Directory, try out the new RunwayML nodes, and accelerate your creative AI journey today.

‍

© Griptape, Inc
Sitemap