Lock in $30 Savings on PRO—Offer Ends Soon! ⏳

Case Study for Repurposing Video Content With G...

Serverless Operations
October 08, 2024
140

Case Study for Repurposing Video Content With Generative AI / AWS Community Day Taiwan 2024

Serverless Operations

October 08, 2024
Tweet

More Decks by Serverless Operations

Transcript

  1. Case Study for Repurposing Video Content With Generative AI Kazuki

    Miura Hokkaido Television Broadcasting Co., Ltd. Sonu Kim Serverless Operations, Inc.
  2. Speakers Kazuki Miura Hokkaido Television Broadcasting Co., Ltd. Sonu Kim

    Serverless Operations, Inc. Media-JAWS Organizer AWS User Group - Japan focusing on Media industry
  3. APIs provided by AWS Access to various generative AI models

    Pricing model On-demand by token consumption Provisioned mode Integration convenience with other AWS services Testing convenience with various models Why we use Amazon Bedrock?
  4. A simple Q&A design using Amazon Bedrock AWS SDK (boto3)

    Amazon Bedrock Claude3 AWS Credentials (SigV4) Simple design that calls API provided by Amazon Bedrock AWS SDK is available in the same way as other AWS services
  5. What is Retrieval-Augmented Generation (RAG)? Questioner Ask Answer 1.Similarity search

    2.Extended (Augmented) context Vector store (Knowledge contents) 3.Question+Search +Augmented Context 4.Generated Answer LLM (Text generative AI Model)
  6. Questions about use cases of generative AI Can text generation

    AI help us in our daily work? Can we use it in revenue-generating services?
  7. Exploring use cases for text generative AI Search engine/solutions Internal

    document land history in real estate and construction industries Providing Q&A functionality for customers Offering product-specific Q&A based on user manuals Inquiries from seminar or lecture content Guidance on documentation processes in HR New hire onboarding Year-end tax adjustments Various administrative procedures
  8. What is “SODANE”? Owned media site with 1,000 blog posts

    updated annually Converting video content from one-time broadcasts and cooking shows into blog articles manually Content creation workload 30% of articles written by TV program staff 70% created by the media operations team Managing this could not be easy alongside their primary tasks
  9. What Inspired Us to Use Gen AI for Blog Writing

    After 5 years of blogging and just as ad revenue began to increase, the key team member was reassigned Content creation does not start from a zero-bases, as articles are based on previously aired programs Problem-solving criteria Maintain the pace of blog posting even with new members joining Reduce dependency on specific individuals Establish a systematic approach to media operations
  10. If the article has to be written based on the

    video, It should be possible to transcribe the audio to text (though it is NOT 100% accurate) Could a text-generating AI be used to proofread the transcription, and create a blog post with the content summarized? Our Breakthrough Idea Amazon Transcribe Amazon Bedrock Audio
  11. 1.Upload video file 2.Transcribe 3.Capture images 4. Place images for

    each paragraphs Technical Challenge How to capture screenshots from a video and specify them to be relevant to the article content Video capture images must be placed to match each paragraph of the article as a MUST requirement. ? ? Amazon S3 Amazon Transcribe
  12. Solving Technical Challenge Through Advanced RAG design 1. Take multiple

    captures at intervals of a few seconds in advance 2. Use LLM capable of image input to convert the images into text description and embed them into vector store 3. When creating the article, retrieve image descriptions from vector store, and let the LLM select the specific images to use. Take source video captures at intervals of few seconds Load images Convert to text Save (Embedding) Retrieve text description Generate article
  13. A portion of the actual prompt used (for text conversion

    of captured images) You are an AI assistant designed to provide detailed descriptions of captured images from videos. After describing the image, remember which number this image is and which article ID it is associated with. Also, use numbering notation to clearly indicate which number description this image is. The response must be in the following format: --- Article ID: *Article ID received from the question* Image number: *Image number received from the question* Description: *Description of the image* --- Now, please describe the attached image.
  14. A portion of the actual prompt used (for article generation)

    I would like to request the creation of a blog post with images in Japanese. Please structure the content of the article as a blog post composed of multiple paragraphs, based on the text of the question including transcription content. Each paragraph should be as detailed as possible while remaining easy to read. Please specify the image numbers that correspond to the article ID, matching them to the appropriate content. {context} Please specify the image numbers in ascending order for each paragraph, ensuring there are no duplicates across all paragraphs. Furthermore, you must only use image numbers from the following list. You are not allowed to use any image numbers that are not listed below. {capturedImages} For each paragraph, please indicate the image number using the following URL format. The actual image number will be inserted where it says 【imageNumber】. https://sample.cloudfront.net/{article_id}/【imageNumber】.jpg Lastly, please provide a title for the article. Now, please begin the task. Please write a blog article with the following contents:
  15. Technical Challenge problem-solved design 1.Upload video file 2.Transcribe 3.Capture images

    4. Convert & Embed image as text 5. Retrieve and generate article Amazon S3 Amazon Transcribe Vector store Claude 3
  16. How to use & Outcomes Create 7-8 AI-generated articles in

    advance for each program Review & proofread by writers before updating the blog. Average article creation time reduced by over 50%
  17. Showcased at AWS Summit Tokyo 2024 as a case study

    Case studies of customers using generative AI in production environments