Classification of Moving Visual Media

Classification of Moving Visual Media 
September 2023 Mark Ghuneim  

Why: Our mediascape which was static for decades is now exploding with modalities. 
What: is a survey of moving media types + an attempt at classification. 
How come: This is important because and updated classification framework was needed.
Feedback: RFC (Request For Comments) on the framework

Temporal Media: 
Temporal media encompasses content that evolves over time, providing a dynamic experience.

Mechanical media:
Mechanical media refers to traditional methods of producing and displaying visual content, 
primarily relying on physical mechanisms or electronic devices without advanced computational manipulation.
Film: Traditional cinematic storytelling captured on celluloid, representing the early days of motion pictures.
Video: The electronic successor to film, offering digital recordings for various purposes.
Animation: A convergence of art and technology, presenting moving images through sequential frames.
Stop motion animation: A unique blend of real-world objects and frame-by-frame photography.
Generative media:
Generative media encompasses visual content that is produced or modified using algorithms, often without 
direct human intervention. The term "generative" implies that the content is generated, often in a procedural 
or automatic manner, rather than being directly crafted by hand or traditionally recorded.
Generative video: A marriage between art and algorithms, creating evolving visual spectacles.
Deepfakes/Manipulated media: Offspring of deep learning, enabling face/content replacements in videos.
Synthetic media from prompts: Producing media content from textual or visual instructions.
Interactive Media
This media type hinges on user participation, reacting or evolving based on user actions.
Virtual Reality (VR) Simulated experiences, either mirroring or diverging from reality.
Augmented Reality (AR) Enhanced version of reality, overlaying digital information on the physical world.
Mixed Reality (MR) A hybrid realm where the digital and physical worlds coalesce.
Vector-based media: 
Non-Temporal Media with Potential for Motion - Though potentially static, this media type can be animated or 
made dynamic.
Vector Graphics: Mathematical elegance meets art, creating scalable images.
2D Animation using vector graphics: Bringing vector images to life with motion.

Identifying AI-generated images with SynthID

New tool helps watermark and identify synthetic images created by Imagen

A new tool helps watermark and identify synthetic images produced by Imagen.

Whether it’s NIST initiatives or outcomes from the POTUS meeting, watermarks are now in vogue.

“We make new products that comes with risks, we also offer innovative tools to mitigate them.” is some game.  Keeping
watermarks invisible to the human eye seems to be an effective approach – said no-one ever.  Additionally, with walled
gardens intensifying their defenses and shifting liability towards consumers, tools become even more essential.

The open-source market continues to create valuable products and shows no signs of vanishing into thin air.

AI-14 Harmonies + Copyright Law

Earlier this year, I published a list of the 14 modalities of AI in song creation. (below list)
This name was inspired by the idea of harmonies, which are a fundamental aspect of music

Raising this list again because of a recent ruling by a federal judge in Washington, D.C.,
that artwork created by artificial intelligence is not eligible for copyright protection because
it lacks human authorship.

While I am not a lawyer, and don’t play one on TV this ruling seems to answer the
question of whether all AI-created works are ineligible for copyright protection, or if there
is a grey area for works that involve significant human input.

Anyone working with AI and music creation knows that there is a lot of human involvement
in all aspects of the process. Some key takeaways from the recent rulings on copyright protection
for AI-generated works include:

•Copyright law is based on the principle that copyright protection is only granted to works that

are the product of human creativity. In the DC ruling, the judge stated that “human authorship”
is the bedrock of copyright law.

•AI-generated works are not considered to be the product of human creativity, and therefore are
not eligible for copyright protection. The Copyright Office has denied registration for AI-generated

•There is uncertainty about how this principle will apply to AI-generated works that are created
with significant human input.

These recent rulings suggest that the courts are still trying to reckkon with the challenges posed
by AI-generated works.  As value is created with the media the likely hood of aspects of this being
revisited by the courts is likely.


AI-14 Harmonies: Modalities Involving AI in Song Creation, Including Music Video

1. AI-Written (Music and Lyrics), AI-Sung
2. AI-Written (Music and Lyrics), Human-Sung
3. Human-Written (Music and Lyrics), AI-Sung
4. AI-Written Music, Human-Written Lyrics, Human-Sung
5. AI-Written Music, Human-Written Lyrics, AI-Sung
6. Human-Written Music, AI-Written Lyrics, Human-Sung
7. Human-Written Music, AI-Written Lyrics, AI-Sung
8. AI and Human Collaboration on Writing, Human-Sung
9. AI and Human Collaboration on Writing, AI-Sung
10. AI as a Music Producer (AI creates backing track, human writes lyrics), Human-Sung
11. AI Creates Music Video for Human-Written and Performed Song
12. AI Creates Music Video for AI-Written and Human-Performed Song
13. AI Creates Music Video for Human-Written and AI-Performed Song
14 . AI Creates Music Video for AI-Written and AI-Performed Song


User-agent: GPTBot Disallow: /

In early March of this year I started talking about the importance of implementing “robots.txt” and the “nofollow” attributes because the great AI transcoding was in full effect.

The time has clearly come to use code as a explicit statement around IP
it looks like this :

User-agent: GPTBot
Disallow: /

A Verge piece looks at the NYT implementing this. The New York Times blocks OpenAI’s web crawler / The NYT’s robot.txt page that controls how it appears to automated bots built to index the internet now specifically disallows OpenAI’s GPTBot.



Synthetic city

AI= surveillance, by definition is predatory. This, not that.  This text not that text. this image not that/etc.

That said its there for the subverting. To turn it in on itself, take the distortion,

feedback glitch + transcode modalities to a synthetic video feed we all need.


From Prompts to Outputs: Safeguarding Interests in Generative AI Agreements

Some Thoughts on Generative AI Best Practices…

  • Understand Generative AI:
    • Recognize that generative AI identifies patterns in training data to produce new content based on user prompts.
    • Assess the balance between the vast potential and risks of using generative AI tools.
  • Contractual Clarity:
    • Always ensure your use of a generative AI tool is under a clear agreement. Avoid tools without explicit terms.
    • Consider “enterprise” or “business” versions which might have more favorable terms.
    • Be aware of different service categories, such as plugins or APIs, which might have distinct terms.
  • Rights in Prompts:
    • Confirm who retains ownership rights of prompts submitted to the AI.
    • Be cautious of terms that grant providers rights to use, modify, or distribute prompts, especially if they can use them as training data.
    • Educate employees and contractors about the risks associated with submitting confidential or proprietary data in prompts.
  • Rights in Outputs:
    • Verify ownership rights of the outputs generated by the AI.
    • Ensure providers don’t have undue rights to use, distribute, or modify outputs, especially for valuable intellectual property.
  • Confidentiality Measures:
    • Ascertain if the AI provider is subject to any confidentiality obligations regarding your prompts or outputs.
    • Avoid breaching third-party confidential agreements or privacy laws by inadvertently sharing data with AI providers.
    • Implement a comprehensive generative AI use policy to protect sensitive company information.
  • Indemnities:
    • Understand the indemnification terms. Some providers may offer indemnification against third-party intellectual property claims, while others might not.
    • Be cautious of terms that place broad indemnity obligations on end users.
  • Limitations of Liability:
    • Recognize that many AI providers limit or disclaim liability, especially for indirect damages.
    • Consider enterprise or business account options that might provide more favorable liability terms.
  • Holistic Evaluation:
    • Conduct a comprehensive assessment of both the technical and legal risks associated with generative AI tools.
    • Weigh the benefits against the legal risks, and consider if providers offer more favorable tool versions or contractual terms.

Ed Note: This is not legal advice, I am not a lawyer (hire one!) – Just pointing out areas to investigate. 🙂

Laziness as a Driver in AI: The Interplay of Nudging and Boosting

Nudging, Boosting, and Self-Nudging

Behavioral science interventions, with their diverse nature, can be classified in various ways, such as through UCL’s Behaviour Change Technique Ontology. A defining characteristic is the degree of individual autonomy or agency during behavior change.

The concept of a “nudge” is to subtly alter choice architecture, gently guiding individuals toward certain behaviors. For instance, reframing nurses’ handwashing practices as an act of patient care rather than a compliance task can be seen as a nudge.

These nudges, while effective initially, often need repetition, mirroring how AI systems require ongoing nudges for optimization. This similarity between human and AI behavior can be attributed to an inherent “laziness” or a tendency toward efficiency.

How many times have your generated a script with AI help and the AI just tosses in placeholder code.  “#The rest of your code here” and you have to prompt it to provide the full text. Over and over,  just plain lazy.

Conversely, “boosts” aim to empower individuals with actionable competencies. In the context of nursing, a boost might make the risks of unwashed hands more tangible, translating abstract percentages into more relatable frequencies. These boosts, similar to enhancing AI with comprehensive data or refined algorithms, grant more autonomy and efficiency. Yet, both humans and AI can exhibit a form of “laziness” when over-relying on certain competencies, leading to potential stagnation.

While boosts hold promise, they are particularly effective for individuals with ambiguous or conflicting goals. Nudging such individuals might be ethically complex. Moreover, in “toxic choice environments” – where certain setups are tailored to manipulate choices, such as some fast-food chains or AI-driven platforms – boosting serves as a countermeasure, helping individuals navigate and resist undue influences.

It’s crucial to recognize that nudges and boosts aren’t antagonistic but can synergize. A reminder (a nudge) might prompt the application of a learned skill (a boost). An example includes a ribbon on a car door nudging individuals to use the Dutch Reach technique.

Furthermore, the advent of “self-nudges” allows individuals to tailor their decision-making environments. This empowerment, reminiscent of equipping AI with adaptive learning capabilities, underscores the significance of autonomy and ongoing growth.

This underlying theme of laziness, draws notable parallels between human decision-making and AI system optimization.  Often it is what is needed to get to the finish line. –  This leads to a larger story on reliability and true enterprise utility.


(ed note – AI helped with the examples used in this post, removed a totally unrelated paragraph that ended up in this post)

AI Training data and video conference software

“Just a heads up, Zoom terms now allow training AI on user content with no opt out. (Technically, you can delete your data later, but the training has been done.)

My framing of Zoom has always been: ‘dangerous data collection software posing as a video conferencing service.’

Over the last few years, maintaining any kind of PII operational security has been an intractable problem. Even if you lock down your basic best practices online, it’s only one vector.

My desk has a stack of letters from companies who have been exploited from medical, including biometric data, to financial information – all offering free credit reports. Get your free credit report. Right now, I have enough free credit reporting to last three lifetimes.

The market does not really move forward until users have agency over their own data. AI is blowing through that opportunity moment right now with centralized services and data training and collection vs at the local level, but that’s a long post and another story.

Do not use this service with sensitive information. Business, institutes of higher learning, medical, and therapy, etc. This means you.”