Alternate Text Generation with AI

Overview

The Alternate Text Generation with AI AddOn enables host application end users to instantly generate alternate text for images using AI. This feature is only available for images.

Visit the Alternate Text Generation with AI Knowledge Base article to learn more about the key benefits and use cases of this feature. Continue in this article to learn more about how to implement the Alternate Text Generation with AI AddOn in your host application.

Prerequisites

Prior to getting started with the configuration, ensure you have the following:

• A working subscription with Azure

• Superpower or Enterprise plan

Configuration Steps

This section discusses the steps you need to take to configure alternate text generation in your host application.

Developer Console

To active alternate text generation with AI in your Beefree SDK Developer Console, take the following steps:

1. Go to the Beefree SDK Developer Console

2. Enter your application

3. Click on “AddOns”

4. Select the “Azure AI Vision – Image Analysis” AddOn

5. Click “Install”

   Once you click “Install”, the “Set Up Content” pop up will appear to configure the AddOn

6. Toggle “Enable” to on

7. Insert your Azure API Key and Endpoint

Note: If you do not yet have an Azure API Key and Endpoint, navigate to the Microsoft Azure Configuration Steps of this guide.

Microsoft Azure Configuration Steps

Azure Computer Vision is a key component of the Alternate Text Generation with AI AddOn. The steps highlighted in this section will guide you through how to obtain your Computer Vision or Cognitive Services API Key and Endpoint. If you have already obtained both, you can skip this section and navigate to the Limit the Usage section of this guide to learn more about customizing the AddOn for your end users.

Take the following steps to configure Azure Computer Vision and successfully integrate the AddOn into your host application:

1. Login to the Microsoft Azure Portal

   1. Go to “Azure services”

   2. Click “Create a resource”

   3. Search “Cognitive Services”

   4. Select “Cognitive Services” or “Computer Vision”

   Note: Cognitive Services allows you to use other functions of Azure AI, but either “Cognitive Services” or “Computer Vision” will work for configuring the alternate text generation. Ensure you select the feature that is best for your use case.

2. Click “Create”

   It will ask you for “Project Details”

3. Complete the following information to satisfy the “Project Details” requirement:

• Subscription

• Resource group

• Instance Details

• Region

• Name

• Pricing tier

Note: Version 4.0 is not yet available in all regions, but it is required for this feature to work. If you only have access to the 3.2 version, this feature will not work. Ensure that you have access to the correct version.

1. Click “Review + create” to create the resource

2. Azure will create your resource with an endpoint and API key

3. Input in the endpoint in the provider url field within in the Beefree Developer Console

4. Input in the API key within the API key field within in the Beefree Developer Console

Note: Once you complete these steps, wait at least 15 minutes for the resources to become available. After 15 minutes, you will be able to move forward with using your API Key and Endpoint with the feature.

For more information, visit the Microsoft Azure Computer Vision documentation.

Properties and Parameters

Properties and Parameters	Data Type	Example	Description
handle	String	'ai-alt-text'	Represents the identifier for the specific add-on handling alt text generation.
infoMessage.detail.consumedImages	Number	5	Indicates the number of images processed by the bulk alt text generation call.
imagesCounter	Number	10	Keeps a running total of the number of images that have been processed for alt text generation.
imagesAvailable	Number	100	Indicates the total number of images available for alt text generation based on the user's subscription or quota.
isIconDisabled	Boolean	true	Determines whether the icon for the add-on is disabled, based on whether the imagesCounter has reached the imagesAvailable limit. Default value is false.
isUpsellEnabled	Boolean	true	Determines whether upsell options or notifications should be enabled for the user.
refreshedUsageSettings	Object	{ addOns: [...] }	Contains the updated configuration settings for the add-on, including the current usage metrics and state settings.
bee.loadConfig	Function	bee.loadConfig(refreshedUsageSettings)	Responsible for reloading the configuration settings for the add-on to reflect the latest usage metrics and states.

Limit the Usage

After you complete the configuration, the Alternate Text Generation with AI tool will be available to your end users. At this point in the process, the tool is offered for free to your end users. Take the steps outlined in this section to limit the usage of this feature for your end users.

The onInfo call in the following code snippet updates the usage limits for the Alternate Text Generation with AI feature. When AI-generated alternate text is used, the infoMessage.detail.consumedImages field indicates how many images were processed. This number is added to imagesCounter, which keeps track of the total images used so far. The backend processes images in bulk, so imagesCounter reflects the cumulative count of consumedImages from a single backend call. The refreshed settings ensure that the isIconDisabled property is updated if the image usage limit (imagesAvailable) is reached, and the settings are reloaded with bee.loadConfig(refreshedUsageSettings).

If you’d like to limit the usage of the Alternate Text Generation with AI tool, take the following steps to configure usage limitations.

1. Implement the onInfo call

Sample code onInfo call:

else if (handle === 'ai-alt-text') {
            imagesCounter += infoMessage.detail.consumedImages
            const refreshedUsageSettings = {
              addOns: [
                {
                  id: "ai-alt-text",
                  settings: {
                    imagesAvailable: imagesAvailable,
                    imagesUsed: imagesCounter,
                    isIconDisabled: (imagesCounter >= imagesAvailable) ? true : false,
                    isUpsellEnabled: isUpsellEnabled,
                  }
                },
              ],
            }
            // Reload Config
            bee.loadConfig(refreshedUsageSettings)
          }

1. Confirm the AddOn handle is ai-alt-text

The following sample code shows imagesCounter and imagesAvailable, which determine how many images you have used, and the image limit the host application setup for users, respectively.

let imagesCounter = 0
let imagesAvailable = 5

Note: In this example, the end user can only make five requests in total, after the fifth request, the end user will no longer be able to generate alternate text. The number five is defined in the configuration and can be customized. For example, if imagesAvailable is set ten, the end user will be able to perform ten requests.

1. In the AddOn settings, isIconDisabled is configured to automatically disable the Alternate Text Generation with AI feature when the counter reaches zero. However, through the Token Upselling notification banner, application end users will have the opportunity to purchase more images and regain access to the feature.

  // Check which addon
        const handle = data.detail.handle
        if (handle === 'ai-alt-text') {
          imagesCounter += infoMessage.detail.consumedImages || 1
          const refreshedUsageSettings = {
            addOns: [
              {
                id: "ai-alt-text",
                settings: {
                  imagesAvailable: imagesAvailable,
                  imagesUsed: imagesCounter,
                  isIconDisabled: imagesCounter >= imagesAvailable,
                  isUpsellEnabled: isUpsellEnabled,
                }
              },
            ],
          }
          bee.loadConfig(refreshedUsageSettings)
        }

Display the Usage Widget

To display the widget usage, take the following steps:

1. The call looks for imagesAvailable value and imagesUsed value

2. To activate the display usage widget, provide the usage data via the AddOn settings. Since the editor doesn’t track usage, you’ll need to refresh the values via the bee.loadConfig method to keep the display widget data current

let imagesCounter = 0
let imagesAvailable = 1000
let isUpsellEnabled = true

const beeConfig = {
  uid: 'string',
  ...
  addOns: [
    {
      id: "ai-alt-text",
      settings: {
        imagesAvailable: imagesAvailable,
        imagesUsed: imagesCounter,
        isIconDisabled: false,
        isUpsellEnabled: isUpsellEnabled,
      }
    },
  ],
  ...
  onInfo: function (data) {
    if (data.code === 1000) {
      const handle = data.detail.handle
      
      if (handle === 'ai-alt-text') {
        imagesCounter += data.detail.consumedImages
        const refreshedUsageSettings = {
          addOns: [
            {
              id: "ai-alt-text",
              settings: {
                imagesAvailable: imagesAvailable,
                imagesUsed: imagesCounter,
                isIconDisabled: (imagesCounter >= imagesAvailable),
                isUpsellEnabled: isUpsellEnabled,
              }
            },
          ],
        }
        bee.loadConfig(refreshedUsageSettings)
      }
    }
  },
}

Note: If either of the settings imagesAvailable or imagesUsed are not sent, the application will not show the consumption.

Advanced Permissions

This section discusses advanced permissions for your application users. You can use the two settings IsIconDisabled and enabled to configure permissions for your end users. To configure both booleans, take the steps outlined in the subsequent sections.

Disable Prompts Per User

To disable prompts per user, perform the following:

1. Set isIconDisabled to true

{
                id: "ai-alt-text",
                settings: {
                  isIconDisabled: true
                }

Note: The end user will be able to see the button, but will not be able to use the button through setting the isIconDisabled setting to true

You can reference the difference in appearance in the following images.

Image 1.0 displays an example of the wand when the feature is not active.

Image 2.0 displays an example of the AI wand icon when the feature is active.

Disable AddOn Per User

To disable the AddOn completely for an end user, perform the following:

1. Set the enabled field to false

const beeConfig = {
    uid: 'inactive-user',
    ... 
    addOns: [
      {
        id: "ai-alt-text",
        enabled: false
      },
    ],
    ...
}

Feature Limitations

Ensure you familiarize yourself with the following feature limitations to ensure this AddOn is suitable for your application’s needs:

• You can generate the caption for an image multiple times, but you’ll always get the same result for the alternate text

• The AI model does not work well with PNG or SVG images with transparent backgrounds, especially when the outline is black

• Only available in English

• Not available for Multi-language templates

• Images must have URLs and be publicly accessible

• Only available for static images, not available for dynamic images

• Not available for custom AddOn of image type

Billing

Unlike the OpenAI AddOn, the Alternate Text Generation AddOn does not charge by tokens. Rather, the AddOn charges for each image caption you generate. Reference the Microsoft Azure Pricing page for additional information.

If you want to know if you are being charged, and you have the usage widget enabled, you will only be charged if the counter goes down.

Note: If an error occurs after the icon for the Alternate Text Generation AddOn is clicked by an end user, and text is not generated, you will not be charged. You will only be charged for instances when a caption is successfully generated.

Error Handling

In the event an error occurs, the end user will see an alert and you will receive a message from the Azure API with additional details regarding the error.

For more information on potential errors and error codes, reference the Azure Cognitive Services REST API reference or the Computer Vision REST API reference – Azure Cognitive Services depending on which service you opted in for to configure this feature.

Additional Considerations

The AI Alternate Text Generator is compatible with token upselling. The token upselling feature notifies the end users when their token usage is running low and when it is completely out. Both notifications include an option for the end user to purchase more tokens. The appearance of these notifications are customizable. Read the Token Upselling documentation to learn more about configuring token upselling within your host application.

FAQs

Q: Will Azure keep the photos used with this feature?

A: The following answer is directly from the Azure Data and privacy for Image Analysis website:

The images you submit to Image Analysis service are processed in real time, and the input images and results are not retained or stored in the service after processing.

For more information on image processing and data security, refer to the Azure AI services security documentation.