Learn more about how to configure and use our Azure AI Vision - Image Analysis AddOn.

Overview

The Alternate Text Generation with AI AddOn enables host application end users to instantly generate alternate text for images using AI. This feature is only available for images.

Visit the Alternate Text Generation with AI Knowledge Base article to learn more about the key benefits and use cases of this feature. Continue in this article to learn more about how to implement the Alternate Text Generation with AI AddOn in your host application.

Prerequisites

Prior to getting started with the configuration, ensure you have the following:

• A working subscription with Azure

• Superpower or Enterprise plan

Configuration Steps

This section discusses the steps you need to take to configure alternate text generation in your host application.

Developer Console

To active alternate text generation with AI in your Beefree SDK Developer Console, take the following steps:

1. Go to the Beefree SDK Developer Console

2. Enter your application

3. Click on “AddOns”

4. Select the “Azure AI Vision – Image Analysis” AddOn

5. Click “Install”

   Once you click “Install”, the “Set Up Content” pop up will appear to configure the AddOn

6. Toggle “Enable” to on

7. Insert your Azure API Key and Endpoint

Note: If you do not yet have an Azure API Key and Endpoint, navigate to the Microsoft Azure Configuration Steps of this guide.

Microsoft Azure Configuration Steps

Azure Computer Vision is a key component of the Alternate Text Generation with AI AddOn. The steps highlighted in this section will guide you through how to obtain your Computer Vision or Cognitive Services API Key and Endpoint. If you have already obtained both, you can skip this section and navigate to the Limit the Usage section of this guide to learn more about customizing the AddOn for your end users.

Take the following steps to configure Azure Computer Vision and successfully integrate the AddOn into your host application:

1. Login to the Microsoft Azure Portal

   1. Go to “Azure services”

   2. Click “Create a resource”

   3. Search “Cognitive Services”

   4. Select “Cognitive Services” or “Computer Vision”

   Note: Cognitive Services allows you to use other functions of Azure AI, but either “Cognitive Services” or “Computer Vision” will work for configuring the alternate text generation. Ensure you select the feature that is best for your use case.

2. Click “Create”

   It will ask you for “Project Details”

3. Complete the following information to satisfy the “Project Details” requirement:

• Subscription

• Resource group

• Instance Details

• Region

• Name

• Pricing tier

Note: Version 4.0 is not yet available in all regions, but it is required for this feature to work. If you only have access to the 3.2 version, this feature will not work. Ensure that you have access to the correct version.

1. Click “Review + create” to create the resource

2. Azure will create your resource with an endpoint and API key

3. Input in the endpoint in the provider url field within in the Beefree Developer Console

4. Input in the API key within the API key field within in the Beefree Developer Console

Note: Once you complete these steps, wait at least 15 minutes for the resources to become available. After 15 minutes, you will be able to move forward with using your API Key and Endpoint with the feature.

For more information, visit the Microsoft Azure Computer Vision documentation.

Properties and Parameters

Limit the Usage

After you complete the configuration, the Alternate Text Generation with AI tool will be available to your end users. At this point in the process, the tool is offered for free to your end users. Take the steps outlined in this section to limit the usage of this feature for your end users.

The onInfo call in the following code snippet updates the usage limits for the Alternate Text Generation with AI feature. When AI-generated alternate text is used, the infoMessage.detail.consumedImages field indicates how many images were processed. This number is added to imagesCounter, which keeps track of the total images used so far. The backend processes images in bulk, so imagesCounter reflects the cumulative count of consumedImages from a single backend call. The refreshed settings ensure that the isIconDisabled property is updated if the image usage limit (imagesAvailable) is reached, and the settings are reloaded with bee.loadConfig(refreshedUsageSettings).

If you’d like to limit the usage of the Alternate Text Generation with AI tool, take the following steps to configure usage limitations.

1. Implement the onInfo call

Sample code onInfo call:

else if (handle === 'ai-alt-text') {
            imagesCounter += infoMessage.detail.consumedImages
            const refreshedUsageSettings = {
              addOns: [
                {
                  id: "ai-alt-text",
                  settings: {
                    imagesAvailable: imagesAvailable,
                    imagesUsed: imagesCounter,
                    isIconDisabled: (imagesCounter >= imagesAvailable) ? true : false,
                    isUpsellEnabled: isUpsellEnabled,
                  }
                },
              ],
            }
            // Reload Config
            bee.loadConfig(refreshedUsageSettings)
          }

1. Confirm the AddOn handle is ai-alt-text

The following sample code shows imagesCounter and imagesAvailable, which determine how many images you have used, and the image limit the host application setup for users, respectively.

let imagesCounter = 0
let imagesAvailable = 5

Note: In this example, the end user can only make five requests in total, after the fifth request, the end user will no longer be able to generate alternate text. The number five is defined in the configuration and can be customized. For example, if imagesAvailable is set ten, the end user will be able to perform ten requests.

1. In the AddOn settings, isIconDisabled is configured to automatically disable the Alternate Text Generation with AI feature when the counter reaches zero. However, through the Token Upselling notification banner, application end users will have the opportunity to purchase more images and regain access to the feature.

  // Check which addon
        const handle = data.detail.handle
        if (handle === 'ai-alt-text') {
          imagesCounter += infoMessage.detail.consumedImages || 1
          const refreshedUsageSettings = {
            addOns: [
              {
                id: "ai-alt-text",
                settings: {
                  imagesAvailable: imagesAvailable,
                  imagesUsed: imagesCounter,
                  isIconDisabled: imagesCounter >= imagesAvailable,
                  isUpsellEnabled: isUpsellEnabled,
                }
              },
            ],
          }
          bee.loadConfig(refreshedUsageSettings)
        }

Display the Usage Widget

To display the widget usage, take the following steps:

1. The call looks for imagesAvailable value and imagesUsed value

2. To activate the display usage widget, provide the usage data via the AddOn settings. Since the editor doesn’t track usage, you’ll need to refresh the values via the bee.loadConfig method to keep the display widget data current

let imagesCounter = 0
let imagesAvailable = 1000
let isUpsellEnabled = true

const beeConfig = {
  uid: 'string',
  ...
  addOns: [
    {
      id: "ai-alt-text",
      settings: {
        imagesAvailable: imagesAvailable,
        imagesUsed: imagesCounter,
        isIconDisabled: false,
        isUpsellEnabled: isUpsellEnabled,
      }
    },
  ],
  ...
  onInfo: function (data) {
    if (data.code === 1000) {
      const handle = data.detail.handle
      
      if (handle === 'ai-alt-text') {
        imagesCounter += data.detail.consumedImages
        const refreshedUsageSettings = {
          addOns: [
            {
              id: "ai-alt-text",
              settings: {
                imagesAvailable: imagesAvailable,
                imagesUsed: imagesCounter,
                isIconDisabled: (imagesCounter >= imagesAvailable),
                isUpsellEnabled: isUpsellEnabled,
              }
            },
          ],
        }
        bee.loadConfig(refreshedUsageSettings)
      }
    }
  },
}

Note: If either of the settings imagesAvailable or imagesUsed are not sent, the application will not show the consumption.

Advanced Permissions

This section discusses advanced permissions for your application users. You can use the two settings IsIconDisabled and enabled to configure permissions for your end users. To configure both booleans, take the steps outlined in the subsequent sections.

Disable Prompts Per User

To disable prompts per user, perform the following:

1. Set isIconDisabled to true

{
                id: "ai-alt-text",
                settings: {
                  isIconDisabled: true
                }

Note: The end user will be able to see the button, but will not be able to use the button through setting the isIconDisabled setting to true

You can reference the difference in appearance in the following images.

Image 1.0 displays an example of the wand when the feature is not active.

Image 2.0 displays an example of the AI wand icon when the feature is active.

Disable AddOn Per User

To disable the AddOn completely for an end user, perform the following:

1. Set the enabled field to false

const beeConfig = {
    uid: 'inactive-user',
    ... 
    addOns: [
      {
        id: "ai-alt-text",
        enabled: false
      },
    ],
    ...
}

Feature Limitations

Ensure you familiarize yourself with the following feature limitations to ensure this AddOn is suitable for your application’s needs:

• You can generate the caption for an image multiple times, but you’ll always get the same result for the alternate text

• The AI model does not work well with PNG or SVG images with transparent backgrounds, especially when the outline is black

• Only available in English

• Not available for Multi-language templates

• Images must have URLs and be publicly accessible

• Only available for static images, not available for dynamic images

• Not available for custom AddOn of image type

Billing

Unlike the OpenAI AddOn, the Alternate Text Generation AddOn does not charge by tokens. Rather, the AddOn charges for each image caption you generate. Reference the Microsoft Azure Pricing page for additional information.

If you want to know if you are being charged, and you have the usage widget enabled, you will only be charged if the counter goes down.

Note: If an error occurs after the icon for the Alternate Text Generation AddOn is clicked by an end user, and text is not generated, you will not be charged. You will only be charged for instances when a caption is successfully generated.

Error Handling

In the event an error occurs, the end user will see an alert and you will receive a message from the Azure API with additional details regarding the error.

For more information on potential errors and error codes, reference the Azure Cognitive Services REST API reference or the Computer Vision REST API reference – Azure Cognitive Services depending on which service you opted in for to configure this feature.

Additional Considerations

The AI Alternate Text Generator is compatible with token upselling. The token upselling feature notifies the end users when their token usage is running low and when it is completely out. Both notifications include an option for the end user to purchase more tokens. The appearance of these notifications are customizable. Read the Token Upselling documentation to learn more about configuring token upselling within your host application.

FAQs

Q: Will Azure keep the photos used with this feature?

A: The following answer is directly from the Azure Data and privacy for Image Analysis website:

The images you submit to Image Analysis service are processed in real time, and the input images and results are not retained or stored in the service after processing.

For more information on image processing and data security, refer to the Azure AI services security documentation.

Overview

Bulk Alt text Generation with AI is a feature that works as an extension to our existing Alt-text generation with AI feature. The new functionality enables you to provide your end users with an easy method to create alt text captions across multiple image module types simultaneously using artificial intelligence. Through this feature, your application end users will be able to boost the accessibility of their content while also saving a significant amount of time.

Prerequisites

Prior to configuring this feature, ensure that you have the following:

• Superpowers or Enterprise account

• Enabled Azure Vision AddOn in your Developer Console

Default Contents

If you have a default alt text set, the image won't be included among the ones that need an AI-generated alt text. If you have a default value set in your defaultContents, this will require the end user to delete the default text before being able to generate the new alt text with AI. To avoid this behavior, and for this feature to work best, we recommend you do not set a default value for alt text in your beeConfig.

If you do have a default value set, you can delete it one of the following ways:

• Set the alt property as an empty string

• Delete the line of code from your defaultContents

The following code shows an example of the alt property within the defaultContents. In the scenario presented in the following code, we recommend deleting the Default alt value set within the alt property.

var beeConfig = {
  ...,
  defaultContents: {
    ...,
    image: {
      alt: "Default alt value",
      // Add any other default properties as needed
    },
    // Add other default content settings if necessary
  },
  ...,
};

The following code shows an example of the alt property set as an empty string. This is how we recommend you set your code for this feature to work best.

var beeConfig = {
  ...,
  defaultContents: {
    ...,
    image: {
      alt: "",
      // Add any other default properties as needed
    },
    // Add other default content settings if necessary
  },
  ...,
};

Note: You can still use the defaultContents alt value and the AI-generated Alt text in Bulk at the same time. However, it will require the end user to delete the default alt text prior to using the AI-generation tool for that particular image module.

Configuration Steps

To configure Bulk Alt Text Generation with AI for your application, take the Configuration Steps outlined in our Alternate Text Generation with AI documentation.

Token Upselling Compatibility

AI Alt Text Generation in Bulk is compatible with Token Upselling. Through Token Upselling, you can configure your application to verify that end users have enough tokens to generate alt-text for multiple images at once.

The type of images they can create alt-text for are the following:

• Images

• Stickers

• Icons

• GIFs

• Custom AddOn Images

If they do not have sufficient image tokens in their account, you can redirect them to purchase additional image tokens prior to performing the bulk generation.

Multi-Language Templates Compatibility

This feature is not compatible with Multi-language Templates, and is currently only available in English.

Troubleshooting

If you experience any issues with setting up this AddOn, take the following measures to troubleshoot your configuration:

• Ensure your Azure API key is connected correctly within your Beefree SDK Developer Console.

• Ensure your custom endpoint is set up correctly in the SDK console.

• Repeat the configuration steps outlined in this document.

Additional Considerations

As you configure this feature, ensure you consider the following:

• You will be billed through your Microsoft Azure account for AI generation-related features. Ensure you consult their billing page for details on pricing.