Dan York

4:06 am

Jenna (our 16yo miniature poodle) didn’t get the memo about the time change. Can someone remind me why we do this again?

I was going to write about Mozilla and their plans with Mozilla.social this morning… but in trying out WordPress as a platform for a combined newsletter and blog, well… there’s something bright and shiny in the interface… WordPress’ experimental “AI Assistant”. A quick tap and there’s a block added to the WordPress editor that offers a chat interface, along with a warning:

Well, okay, time to play! This feels like the kind of exploration I should livestream to my Twitch account, but it’s 5:00am and other people are asleep in my house – and would you really want to see me at this time?

Maybe I will another time, but right now, the prompt beckons! Let’s see what we can do with this.

[For those unfamiliar with WordPress terminology, “Jetpack” is the set of extra features that Automattic makes available outside the core WordPress functions and features.

Also, the rest of this message is a long walk-through of me experimenting with different AI prompts with then a summary at the end. If you are not interested in generative AI tools and what they can – or cannot – do…. well, you might want to just move on to your next email. ]

Writing a post

First prompt: “write 3 paragraphs about the capabilities of Jetpack AI”

[Note: I am adding “Begin AI Text” and “End AI Text” separators for the purposes of this post. The tool itself just inserts the text directly in the post. There’s no way you could know what is or is not AI-generated. Well… except that the text below is completely different from my style! ]

—— Begin AI Text ——

Jetpack AI, the experimental AI-powered Gutenberg Block offered by WordPress, brings a plethora of capabilities to enhance content creation. First and foremost, it offers seamless customization and generation, enabling users to elevate their writing to new levels. Whether you need assistance brainstorming ideas, organizing your thoughts, or shaping the overall structure of your content, Jetpack AI is there to lend a helping hand. Its advanced algorithms analyze your input and offer insightful suggestions to improve the quality and coherence of your writing.

Furthermore, Jetpack AI excels in generating content tailored to your needs. With its text generation capabilities, you can effortlessly create engaging introductions, compelling conclusions, or captivating anecdotes. Need to fill in the gaps or expand on a particular topic? Jetpack AI can generate coherent paragraphs that seamlessly integrate with your existing content, saving you time and effort. By harnessing the power of artificial intelligence, Jetpack AI empowers content creators with a versatile tool that enhances their writing process.

Moreover, Jetpack AI encourages feedback and continuous improvement. As an evolving tool, occasional imprecision may occur, but the team behind Jetpack AI is committed to refining and honing its capabilities. Your feedback is invaluable in the quest to enhance the accuracy and efficacy of the AI assistant. By actively engaging with users, Jetpack AI aims to fine-tune its algorithms and provide an increasingly reliable and efficient content creation experience. With its endless potential and commitment to improvement, Jetpack AI is undoubtedly a game-changer in the world of content generation.

—— End AI Text ——

Well, okay, that’s a bit over the top in terms of marketing text, i.e. “undoubtedly a game-changer”, but hey, it got me my 3 paragraphs.

My newsletter is now done! Well, no, but I suppose it could be if I just wanted to spew out content.

For those curious, the Jetpack AI tool generates the text in a box and then offers you the chance to accept it or send in another prompt.

After you press “Accept” the text is just inserted into the post.

Let’s try something else…

Translations

Second prompt: “write 50 words in German about Jetpack AI Assistant”

—— Begin AI Text ——

Jetpack AI Assistant, der experimentelle AI-gesteuerte Gutenberg Block von WordPress, bringt eine Vielzahl an Fähigkeiten mit, um die Content-Erstellung zu verbessern. Mit seinen fortschrittlichen Algorithmen und Textgenerierungsfunktionen ermöglicht Jetpack AI nahtlose Anpassungen und hilft dabei, qualitativ hochwertigen und zusammenhängenden Text zu generieren. Eine bahnbrechende Unterstützung für Autoren und Blogger.

—— End AI Text —-

Not bad! The marketing is still strong but then again, I *am* asking about the product. (And if you don’t know German, it’s essentially a summary of the English text earlier, just with those beautiful ginormously long words that German has. I mean… “Textgenerierungsfunktionen”!! )

This could be interesting if you wanted to have a multilingual newsletter, or inject some content in another language.

Summary of a article

So let’s see if it can reach out to the current web. Just this morning I was reading a “lessons learned” post about the attempt to create a Twitter replacement called first T2 then Pebble. Let’s see how it does.

Prompt: Summarize https://medium.com/gabor/from-t2-to-pebble-the-rise-challenges-and-lessons-of-building-a-twitter-alternative-553652f1d1e7 in 5 bullets

—— Begin AI Text (with my emphasis added) ——

• The article discusses the journey of creating a Twitter alternative called T2 and later Pebble.
• It highlights the challenges faced during the development process.
• The article explores the rise and fall of the project, shedding light on its lessons learned.
• Various aspects of building and maintaining a social media platform are discussed.
• The author seeks further research and information in the area of creating alternative social media platforms.

—— End AI Text ——

Interesting. Super high level and kind of bland, but 4 of the 5 bullets are reasonably accurate. The fifth bullet is wrong. Unless I’ve missed it (keeping in mind it is early and I haven’t finished my first cup of tea), I don’t see that anywhere in the article. The author doesn’t say that he’s seeking further research or info. He’s reporting on what they did.

So this is that “occasional imprecision may occur” that was warned about in the block error message.

Otherwise known as “completely making stuff up”

The key points, though, are that:

• The “Jetpack AI Assistant” has access to the current web.
• It can do summaries

Summary of an article in another language

So… what about summarizing an article in another language?

Prompt: “Summarize https://www.spiegel.de/netzwelt/web/mark-zuckerberg-meta-chef-erleidet-kreuzbandriss-beim-kampfsporttraining-a-3ab86c12-de99-497f-850c-85939daef734 in English in 50 words”

—— Begin AI Text —-

Mark Zuckerberg, CEO of Meta, sustained a knee injury, specifically a torn cruciate ligament, while practicing mixed martial arts. The incident occurred during his training session and has raised concerns about his physical well-being. The injury may potentially impact Zuckerberg’s ability to carry out his duties as the head of the company.

—— End AI Text ——

Well, okay, it’s a summary… but it completely misses the point of the article! Specifically that Mark Zuckerberg will have to delay the much-hyped (and never scheduled) fight with Elon Musk.

Oops… that’s kind of the reason someone at Spiegel wrote this click-bait piece!

And that last sentence, “The injury may potentially…”, is not anywhere in the article. Granted, my German is a bit rusty… but I don’t get that sense at all. I mean, it is a logical conclusion to draw, and it’s true – the injury may impact Zuckerberg’s ability to do his job – it’s just not mentioned in the article. The text is all about him not being able to do his fight with Elon Musk.

Well, I hope Mark Zuckerberg does get better, but meanwhile, let’s try a PDF…

Summary of a PDF

So… there’s a public PDF that I know very well. Let’s see how it does on the Internet Society’s 2022 document about Low Earth Orbit (LEO) satellites.

Prompt: Summarize https://www.internetsociety.org/wp-content/uploads/2022/11/Perspectives-on-LEO-Satellites.pdf in 5 bullets

—— Begin AI Text (with my emphasis added) ——

• The document provides perspectives on Low Earth Orbit (LEO) satellites.
• It discusses the benefits and challenges of deploying LEO satellites for internet connectivity.
• The document explores the role of LEO satellites in bridging the digital divide and providing internet access to remote areas.
• It highlights the potential of LEO satellites in supporting emerging technologies such as Internet of Things (IoT) and 5G.
• The document emphasizes the need for collaboration among stakeholders to address regulatory, technical, and governance issues related to LEO satellite deployment.

—— End AI Text ——

Again, okay for a high level and rather bland summary. As an editor I would probably merge the first two bullets. I also would not have so many bullets that start with “The document…”

But for someone who hasn’t read the 16-page PDF, it does get the overall gist of the document.

Except, of course, for the fourth bullet which is completely made up!

I know for a fact (and just double-checked with a search in the PDF) that we never mentioned 5G anywhere in the document. Not even in footnotes. I did the final edits on the doc, so I know it extremely well.

And, we only mentioned IoT in passing twice. We certainly do NOT “highlight” the potential of LEO satellites to support IoT and 5G. It’s perhaps a logical thing that could be included based on info that the large language model (LLM) has in its data, but it’s not in THIS article.

Total FAIL on that bullet point.

So let’s see what else this can do…

Create an article outline

Let’s say that I want to use this to help generate ideas for how to structure an article.

Prompt: “Create an outline for an article about the 5 biggest cities in Vermont”

—— Begin AI Text ——

• Introduction
  • Briefly introduce Vermont as a state known for its natural beauty and charming towns.
  • Mention the significance of exploring the urban side of the state by delving into its five largest cities.
• City 1: Burlington
  • Provide an overview of Burlington as the largest city in Vermont.
  • Highlight key features, such as Lake Champlain, the Church Street Marketplace, and the vibrant arts scene.
  • Discuss the city’s thriving food and craft beer culture.
• City 2: South Burlington
  • Introduce South Burlington as Burlington’s neighboring city and the second-largest in Vermont.
  • Discuss the city’s suburban charm and family-friendly atmosphere.
  • Highlight attractions like the University Mall and the Red Rocks Park.
• City 3: Rutland
  • Describe Rutland as the third-largest city known for its rich history and heritage.
  • Expound on its architectural gems, such as the Chaffee Art Center and the Paramount Theatre.
  • Discuss Rutland’s outdoor recreational opportunities, including nearby ski resorts.
• City 4: Barre
  • Discuss Barre as a unique city known for its granite industry and artistic traditions.
  • Explore the famous Rock of Ages granite quarry and the Vermont Granite Museum.
  • Highlight the city’s vibrant art community, with its numerous public sculptures and art galleries.
• City 5: Montpelier
  • Present Montpelier as the capital and the smallest of the five cities.
  • Discuss its historical significance as the only state capital without a McDonald’s.
  • Highlight attractions like the Vermont State House and the artsy downtown area.
• Conclusion
  • Summarize the unique characteristics of each of the five largest cities in Vermont.
  • Encourage readers to explore and appreciate the diverse experiences that these urban centers offer in the midst of Vermont’s natural beauty.

—— End AI Text ——

Wow! Okay, that’s not bad! That’s a decent framework for writing an article about our five largest cities. (Keeping in mind that this IS Vermont, so our definition of a “city” is a bit smaller than other places.)

And it had never occurred to me that Montpelier doesn’t have a McDonalds. I’m not sure that rises to the level of “historical significance” but hey, I guess that’s a point you could use. (And I’ll note that there IS a McDonalds just over the Montpelier line in Barre, VT, about 8 or 9 minutes from the downtown, so it’s not like people in Montpelier can’t get their Big Macs if they want them!)

But overall that seems a decent outline. I don’t see anything blatantly false.

What else can this thing do?

My tea is getting cold and I realized that while I had paid attention a while back when Jetpack AI Assistant was released, I hadn’t actually read what they said it can do, so let’s check it out: https://jetpack.com/ai/

What else did I not try in this post today?

• Help create forms, tables and lists (hmm… this could be useful!)
• Improving the quality and clarity of your text
• Generating text in different tones, including “Formal, Optimistic, Emphatic, Passionate, or Humorous”
• Generate ideas for titles and headings

I think I have the “free” version and so it says I have access to these features:

• Quota of 20 requests
• Prompt based content generation
• Generate text, tables, and lists
• Adaptive Tone Adjustment
• Superior Spelling and Grammar Correction
• Title & summary generation

Ah, and from reading the FAQ this is 20 requests, period. After that you have to upgrade to use it more. So I am already 7 requests toward that limit. (The 6 in this article plus one more I did.) Now, I *am* on the hosted WordPress.com service and so I don’t know if that limit applies but maybe it does.

Regardless, if I found this AI Assistant useful, the upgrade is only $8.33/month (i.e. $100 / year.)

Final Thoughts

So.. after all these tests, what do I think? Would I use it? Would I pay?

It’s been fun to play with this morning – something entertaining for a Sunday morning while drinking tea.

I think my answer is … I don’t know.

I could see the benefit if I was more frequently creating content. If I needed to create daily posts, or multiple posts per day, I could see it being useful.

I do like the “create outline” exercise I did above. That provided a decent structure and points to think about. And… in developing the article I would discover if any points of the outline were false.

I think that is my one concern with this – and all the other generative AI tools – the fact that completely made-up information is presented plausibly and authoritatively.

Someone who never read the LEOs PDF I referenced might understandably think that we highlighted the uses of IoT and 5G – neither of which we actually did.

In the warning I mentioned at the beginning, Automattic warns that “occasional imprecision may occur”. But this isn’t a case of not being precise, it is outright falsehood. Again, no mention of 5G at all in the document.

I worry that people in a hurry or just looking to slam out content will just use this kind of text without any checking… and the trustworthiness of the Web will decline a bit more as bogus information gets circulated.

Now, this isn’t an issue unique to the JetPack AI Assistant, it’s across pretty much ALL of the generative AI tools.

And Automattic does warn you from the start, and each time you press the “Accept” button, there is a line below that that says “Automatically generated content may be inaccurate or biased.”

Automattic also flags this AI assistant block as “Experimental” and notes that the tool is “evolving”.

You are definitely warned.

But many will just blindly plunge ahead because the convenience is so real and it’s so incredibly easy.

Regardless, I can see the potential here. Just as we’ve grown used to autocorrect (and joke about the occasional mishap), I suspect we’ll grow use to having these kind of tools around and over time they’ll just become part of our daily workflow.

And I’m sure the tools WILL evolve and get better. We’ve already seen great leaps in capabilities over the past year. We’re really still in the early days of people seeing what these LLMs are truly capable of.

For my part… well… I guess I have 13 more queries before I have to start paying, so I’ll probably experiment some more.

Yikes… this was a long one! If you read this far I hope you found it interesting. I would be very curious to know:

• What do YOU think about this tool?
• Where could you see yourself using it? (Are you already using it?)
• Have you incorporated another AI tool in your own workflow?

Drop me a note in email – if you are a subscriber, you should just be able to reply back. And if you aren’t a subscriber, just hit this button and you’ll get future versions.

This IS also a WordPress hosted blog, so you can visit the main site and add a comment to this post, like we used to do back in glory days of blogging.

Or if you don’t want to do email, send me a message on one of the various social media services where I’ve posted this. (My preference continues to be Mastodon, but I do go on others from time to time.)

And yes, this was totally something I should have streamed to Twitch… maybe I still will, and use these examples and then explore some of the things that I didn’t dive into. Follow me over there if you want to know if I go live and do this.

Still wondering about why we need to do this time change,
Dan

P.S. As I started the “Publish” process, I noticed in the sidebar that the AI Assistant can “Check for mistakes and verify the tone of your post before publishing”. Not something I’m going to do for THIS post, but maybe I’ll do that for future posts or the Twitch stream.

Connect

Disclaimer

Disclaimer: This newsletter is a personal project I’ve been doing since 2007 or 2008, several years before I joined the Internet Society in 2011. While I may at times mention information or activities from the Internet Society, all viewpoints are my personal opinion and do not represent any formal positions or views of the Internet Society. This is just me, saying some of the things on my mind.