Features > Stable Diffusion Integration with RunPod

There are many ways of deploying Stable Diffusion models through different set of Cloud Providers but with React Native Starter AI, we are deploying Stable Diffusion, the AI image generation model, through RunPod because it is one of the most cost-effective ways of doing it thanks to their Serverless Instances. By serverless instances, we are only paying what we use in seconds.

Running Stable Diffusion on Serverless RunPod Instances

Main deployment steps of Stable Diffusion to RunPod serverless instances is explained in details in AI Services Docs. Here we are going to dive deep into how we are using the Stable Diffusion API for generating images.

Text To Image Flow

In text to image generation flow, we are using Stable diffusion deployed on a RunPod serverless instance. The RunPod API token is secured in Firebase functions backend as explained in AI Services Docs. The image generation code is just like below:

react-native-starter-backend/functions/src/index.ts

  logger.info("Started text to image", { structuredData: true });
  const requestBody = request.body;

  const style = requestBody.style;
  const prompt = requestBody.prompt;

  const aiRequestBody = constructTextToImageBody({ prompt, style });

  const config = {
    method: "post",
    maxBodyLength: Infinity,
    url: process.env.RUNPOD_TEXT_TO_IMAGE_ENDPOINT,
    headers: {
      Authorization: `Bearer ${process.env.RUNPOD_API_TOKEN}`,
      "Content-Type": "application/json",
    },
    data: aiRequestBody,
  };
  logger.info(`Starting to hit RunPod`, { structuredData: true });
  axios.interceptors.response.use(
    function (response) {
      // Convert status code to 500 if 200 RESPONSE RETURNS "FAILED" or "IN_QUEUE" status to retry
      if (
        response.data.status === "FAILED" ||
        response.data.status === "IN_QUEUE"
      ) {
        response.status = 500;
        throw response;
      }
      response.status = 200;
      return response;
    },
    function (error) {
      return error;
    }
  );

  // Custom retry delay
  axiosRetry(axios, {
    retryDelay: (retryCount) => {
      return retryCount * 1;
    },
    retries: 15,
  });

  const result = await axios.request(config);
  logger.info(`RunPod response is successful!`, { structuredData: true });
  response.json({ result: result.data });

The text to image generation endpoint gets 2 input arguments as style and prompt to be used in image generation flow but these can be extended depending on your needs. The example request body of the endpoint is just like below:

{
  "style": "3d",
  "prompt": "Astronaut on the sun",
}

Image to Image Flow

Image to image flow is also managed through Stable Diffusion AI model which is deployed on RunPod serverless instance. The same as Text to Image flow, the RunPod API key is secured through doing to image generation in Firebase functions backend. The image to image generation code is just like below:

react-native-starter-backend/functions/src/index.ts

// Image to image
export const imageToImage = onRequest(async (request, response) => {
  logger.info("Started image to image", { structuredData: true });
  const requestBody = request.body;

  const style = requestBody.style;
  const prompt = requestBody.prompt;
  const inputImg = requestBody.inputImg;

  const aiRequestBody = constructImageToImageBody({ prompt, style, inputImg });

  const config = {
    method: "post",
    maxBodyLength: Infinity,
    url: process.env.RUNPOD_TEXT_TO_IMAGE_ENDPOINT,
    headers: {
      Authorization: `Bearer ${process.env.RUNPOD_API_TOKEN}`,
      "Content-Type": "application/json",
    },
    data: aiRequestBody,
  };
  logger.info(`Starting to hit RunPod`, { structuredData: true });
  axios.interceptors.response.use(
    function (response) {
      // Convert status code to 500 if 200 RESPONSE RETURNS "FAILED" or "IN_QUEUE" status to retry
      if (
        response.data.status === "FAILED" ||
        response.data.status === "IN_QUEUE"
      ) {
        response.status = 500;
        throw response;
      }
      response.status = 200;
      return response;
    },
    function (error) {
      return error;
    }
  );

  // Custom retry delay
  axiosRetry(axios, {
    retryDelay: (retryCount) => {
      return retryCount * 1;
    },
    retries: 15,
  });

  const result = await axios.request(config);
  logger.info(`RunPod response is successful!`, { structuredData: true });
  response.json({ result: result.data });
});

The image to image generation endpoint gets 3 input arguments as style, prompt and inputImg to be used in image generation flow but these can be extended depending on your needs. The example request body of the endpoint is just like below:

{
  "style": "3d",
  "prompt": "Astronaut on the sun",
  "inputImg": "<BASE_64_IMG>"
}