ai workflows

3 Problems with A.I. Workflows for Content Creation

John Barker

 A.I. Content Creation Promises

Artificial Intelligence (AI) has revolutionized the way we work, especially when it comes to repetitive tasks and workflows. AI has also been touted as a job killer.  As AI improves there will be a change to the many mundane, repetitive job duties. Those will become totally automated.   I even wrote an article last year for the Free Lance Star on AI that you can check out.

I’ve been using AI workflow tools to help with the Titan of Tech Podcast and YouTube channel.  Primarily to help summarize long-form content, create the relevant TL; DR posts, YouTube descriptions, and podcast show notes.

But it’s not been as straightforward as I would like.

The AI Workflow Tools I Use

If you have any task that needs to be repeated, you need to build a workflow.  Your goal should be to get repeatable, accurate, and consistent results each time you run that task.  This is usually referred to as Standard Operating Procedures (SOP).  SOPS should be written in a way that a new person (with some basic knowledge of the required skills), can follow the instructions and get the same results. You want the same desired results over and over again.

On my third podcast edit, it was time to start taking notes to build a workflow.

The two primary tools I use are Descript and ChatGPT version 4.  Descript is a text-based video editor.  It handles the heavy lifting of transcribing the videos for me.  The Descript team has created a custom AI for podcasts, YouTube, and social media inside of the program.

At this point, most people interested in this have already at least tried the free version of ChatGPT.

I started a fresh Word document and began taking notes.  Step 1 through how many ever I needed.  I wanted to copy the exact prompts that got me the best results.

That’s when the problems started to emerge.

Problem 1: Non-Repeatable Prompts with ChatGPT

The Descript AI Workflow tools are in Beta.  I used ChatGPT as the starting point.  My first two podcast edits were great and with very similar results.  I started by going to the previous commands and copying them.  I uploaded the podcast transcript and pasted the command into the box.

And I got totally different results than the first two times.

The show notes I asked for in ChatGPT for the podcast were about a quarter the length of the results in a different format than the first two times I ran the same command.  I didn’t get enough of a summary to be usable.

I repeated the command by saying “do it again”.  I did this command two more times.

I got much better responses to my requests, but the summary sections were still significantly different.  None of it matched my desired results on the first two videos I edited.

I figured I would try the built-in Descript tool.  It was trained for this specific purpose.

Problem 2: Inconsistencies with Descript AI Workflow Tools

I assumed Descript would give me better results.  Many software makers are creating custom AI workflow tools for more specific tasks.

I was wrong.

Descript has a function called Ask AI.  It has built-in AI prompts.  You can use the built-in prompts or modify them.  I decided not to touch the prompt.

The first result I got back was it telling me how AI prompts work.  It didn’t summarize the podcast.  Descript told me how it would go about summarizing the podcast without ever doing it.

I repeated the command and then it would give me a summary.  But of course, in a different format than what I received in ChatGPT.  But I expected that.

I found the Descript AI liked to explain what it would do without doing it.

And just like ChatGPT, using the same command would generate different quality and lengths of responses.

I started going back and forth between Descript AI and ChatGPT.  Then a new issue occurred.

Problem 3:  AI Prioritized the Wrong Information

On at least two occasions when having the AI tools create podcast show notes and a YouTube description the results were just wrong.

In an hour-long conversation, the AI picked up a topic that was discussed for maybe 1 minute and decided the entire conversation revolved around that one topic.  It fleshed out the part of the conversation with its own input.

It built a full outline and bullet points.  It was hundreds of words long.  I ended up deleting those new chat sessions when it started doing this.  It would get stuck on it.

Final Thoughts

I ran more tests. I ended up getting more descriptive with the results I wanted.  Sometimes the AI workflow worked ok.  Sometimes it just ignored my instructions and did what it wanted.

I do not have a prompt that I can reuse that gets me the same results time and time again.  I can see this being a challenge for any type of content creation that relies heavily on AI.

I spent a lot of time early on generating images with AI.  Even my current social media profile picture is AI-generated.  I know if you prompt AI for a picture, the same prompt will give different results.

I was hoping for more consistency with podcast summary tools since I was feeding it so much of the raw information to use for its source material.  I think I was more disappointed with the current version of AI in Descript.  This is part of its job.

Between the two I found myself going back to ChatGPT more frequently.  The results overall were more fleshed out.

I’ve done a lot of complaining here.

These tools were still a gigantic time saver.  Writing a summary for an hour-long conversation is daunting.  Getting chapter timestamps is a huge pain in the ass.  This sped up that process immensely.

BUT!!

You must read the results for accuracy.  You cannot let yourself get on autopilot and just copy and paste the results thinking it’s good.  50-50 chance it’s not.

You can checkout the podcast version of this article here.

You can checkout the video version of this article here.

Or on your favorite Podcast app.

Whenever you’re ready, there is one way I can help you:
I can help if you need help whipping your technology back into shape, looking to improve your cybersecurity, or just need someone to look over shoulder.
Click the above link, send me some details about your business, and schedule a zoom meeting.
Simple as that.

About the Newsletter

Get One Tip every Sunday morning to optimize and secure your business technology.

John Barker

John Barker

John has over 25 years of technology experience and earned a Bachelor’s in Business Management & MBA.  He also holds CISSP and PMP certifications.

0 Comments