Summer Reading

Reading list so far this summer.  (first half of summer )  see  (part 2, end of summer)

All Fours – Miranda July
Co-Intelligence – Ethan Mollusk
The Eye Of The Master
Matteo Pasquinelli
Biography of X Catherine Lacey
Perry Perspective Agents
Daniel Suarez Critical Mass
Charles Duhigg – Supercommunicators
Hari Kunzru Blue Ruin
Kandel -Essays On Art And Science
Tricia Romano – The Freaks Came Out To Wirite
The Twenty Days Of Turin – Giorgio De Maria
Prophet Song – Paul Lynch
Jesmyn Ward Let Us Descend
The Boy, The Mole, The Fox And The Horse – Charlie Mackesy
Candy Darling – Cynthia Carr
Martyr – Kaveh Akbar
The Fraud – Zadie Smith

 

Mediaeater Digest Vol 30, No. 68

NYT workplace advice columnist brings it –  Goodbye Work Friends….”Still, in my heart of hearts, I always wanted to tell you to quit your job. Negotiate for the salary you deserve. Stand up for yourself. Challenge authority. Tell your rude co-worker to shut up. Report your boss to everyone and anyone who will listen. Consult a lawyer. Did I mention quit your job?” …..

When To Write a Simulator “Any problem involving probabilities over time should humble you. Walk away and quietly go write a simulation.”  (sage advice -ed)  

SecondPage – Google Searches without media conglomerates  (tool) this extension blocks most of the top 1000 English sites from appearing in google search. 

Privacy and harm has long been an area of inquiry for me, last week was a bit of a mind blower. Apple put in place secure scaffolding around AI. and understanding (cliaming) your real-time context window with privacy forward AI utility.  

OPSEC win for us all.  Expect many news cycles of Apple needing to put in a back door.  They won’t and never should. Encryption matters and privacy is a fundamental human right. 

Amazon-Powered AI Cameras Used to Detect Emotions of Unwitting UK Train Passengers This falls under the worst possible application of ML.  You do not get to guess (infer intent) what is in people minds and make that actionable!  Full stop.  When did the concept of observation implies supervision and therefore control get lost on society.  We really are getting the governance we deserve. 

While Apple reshaped and recontexualized AI for the marketplace by providing utility around personal privacy,  NSA + OPEN AI have decided the exact opposite optics are needed

Treat your data accordingly and expect information governance, negating which data will-and-won’t, be trained and how your data will and will not be used.  

After Luma dropped their video offering, the Sora halo holding less power,  everyone else is a in constant battle to try and make celloid sense out of latent space.  Here is google paper on Generating audio for video. Every day the range of and tools change and improve. 

Apple / Huggingface repo –  mobile-appropriate vision transformer, CLIP, and image segmentation models 

A.Jafa
A.Jafa

Mediaeater Digest Vol.30, No. 163

States Take Up A.I. Regulation Amid Federal Standstill (nyt) State lawmakers across the country have proposed nearly 400 new laws on A.I. in recent months, according to the lobbying group TechNet. California leads the states with a total of 50 bills proposed, although that number has narrowed as the legislative session proceeds.  (ed note- don’t worry – they will get it done like ad-tech, privacy, social media and other key regulations – dripping sarcasm)

Meta says European data is essential for culturally relevant AI (stackdiary) Meta says it wants to “push to develop AI that understands and reflects European cultures, languages, and humor,” which sounds promising on the surface. However, the approach

Deception abilities emerged in large language models (pnas) This study unravels a concerning capability in Large Language Models (LLMs): the ability to understand and induce deception strategies. As LLMs like GPT-4 intertwine with human communication, aligning them with human values becomes paramount. The paper demonstrates LLMs’ potential to create false beliefs in other agents within deception scenarios, highlighting a critical need for ethical considerations in the ongoing development and deployment of such advanced AI systems. (seems impt -ed)

Private Cloud Compute: A new frontier for AI privacy in the cloud (apple.com) Apple Intelligence is the personal intelligence system that brings powerful generative models to iPhone, iPad, and Mac. For advanced features that need to reason over complex data with larger foundation models, we created Private Cloud Compute (PCC), a groundbreaking cloud intelligence system designed specifically for private AI processing. For the first time ever, Private Cloud Compute extends the industry-leading security and privacy of Apple devices into the cloud, making sure that personal user data sent to PCC isn’t accessible to anyone other than the user — not even to Apple. 

Apple’s On-Device and Server Foundation Models (machinelearning.apple.com) We protect our users’ privacy with powerful on-device processing and groundbreaking infrastructure like Private Cloud Compute. We do not use our users’ private personal data or user interactions when training our foundation models.

MacOS Sequoia to Allow iCloud Logins in Virtual Machines on ARM Macs (developer.apple.com) If someone moves a VM to a different Mac host and restarts it, the Virtualization framework automatically creates a new identity for the VM using the information from the Secure Enclave of the new Mac host. This identity change requires the person using the VM to reauthenticate to allow iCloud to restart syncing data to the VM.

What Apple’s AI Tells Us: Experimental Models⁴ While Apple is building narrow AI systems that can accurately answer questions about your personal data (“tell me when my mother is landing”), OpenAI wants to build autonomous agents that would complete complex tasks for you (“You know those emails about the new business I want to start, could you figure out what I should do to register it so that it is best for my taxes and do that.”). The first is, as Apple demonstrated, science fact, while the second is science fiction, at least for now.

 

(this digest in progress)

Clapper AI
Clapper AI

Generative AI Video: Embracing Interoperability and Open-Source Tools (Clapper AI)

Pointing out an important AI video opportunity: Open source script to screen AI video editor Clapper AI. This project and format is singular in the market. Studios, media companies, producers and talent all stand to benefit by supporting development while it is in the nascent stage.  

It would set a baseline standard thats interoperable and open. It sits atop existing workflows and pipelines  This is a framework that scales for everyone, if done correctly with the added benefit of avoiding costly strategic dependencies in a future timeline.

The ability to switch between different vendors and models seamlessly is a game-changer. This flexibility drives competition like Sora against Veo, StabilityAI against ElevenLabs, and Replicate against Fal.ai, allowing users to harness the best features from each. Unlike traditional open-source tools such as Blender and GIMP, which typically compete against a single commercial product, this tool’s interoperability is a significant competitive edge.

As the media industry is forced to reckon with generative AI multi-model tools, the importance of early stage strategic long term thinking is critical. Be smart, local integration of compute and training into natural production workflows, will fundamentally change how content is created and consumed.

For studios, an open-source script-to-screen AI video editor offers efficiencies + customization and strategic independence. Producers would benefit  interoperability + streamline workflows, allowing the focus to be on storytelling

The barrier removed for talent so they can  prototype their own narratives, better understand set and setting and scene context, dev new story telling skills and act as a catalyst for creative acceleration. Incremental not supplemental.

Interoperable standards can ensure compatibility between different tools and models, enabling seamless workflows and maximizing efficiency, as it should be. (mp3 and audio codecs anyone) It will also foster  innovation and adaptation, but perhaps more importantly avoiding dependencies on specific vendors allows for strategic flexibility. Consider this an open call to action for dev. 

 

 

Links:  – still in dev…

Code  https://github.com/jbilcke-hf/clapper
App   https://clapper.app

 

 

 

Ed note: We are a long, long, long, way from this replacing film making.  This is how we don’t leave that behind.  The list of AI generated multi-model media thats meaningful can be counted on zero fingers right now, be a part of that change.  Fin    

Mediaeater Digest Vol.30, No. 159

YouTube Copyright Transparency Report (youtube) paid out over $70 billion to creators, artists, and media companies in recent years, and it boasts over 100 million Premium and Music subscribers. Its Content ID system, which allows rightsholders to monetize user-generated content, has paid out over $9 billion in ad revenue

Adobe roofies all of their customers (youtube) After they went to the cloud I stopped using their products. -ed

How to Lead an Army of Digital Sleuths in the Age of AI (nyt) Eliot Higgins and his 28,000 forensic foot soldiers at Bellingcat have kept a miraculous nose for truth—and a sharp sense of its limits—in Gaza, Ukraine, and everywhere else atrocities hide online.

Apple to Debut Passwords App in Challenge to 1Password, LastPass  (bloomberg)  Apple Inc. will introduce a new homegrown app next week called Passwords, aiming to make it easier for customers to log in to websites and software, according to people with knowledge of the matter.

Andrew Ng (Twitter) The effort to protect innovation and open source continues. I believe we’re all better off if anyone can carry out basic AI research and share their innovations. Right now, I’m deeply concerned about California’s proposed law SB-1047. It’s a long, complex bill with many parts that require safety assessments, shutdown capability for models, and so on.

FTC Opens Antitrust Probe of Microsoft AI Deal (WSJ) Commission has sent subpoenas to tech giant and startup, asking whether their partnership evaded required government review

Heymusic.ai – Coherence-Oriented Contrastive Learning of Musical Audio Representations

Secret hand gestures in paintings  (pdf) – hand signs in visual art may provide clues about the underlying iconographical symbols. This paper will examine the eventual hidden meanings behind a peculiar hand gesture that has been widely used by several painters

KLING – Chinese new DiT Video AI Generation model 【KLING】 Open access  Generate 120s Video with FPS30 1080P, Understand Physics Better, Model Complex Motion Accurately prompt: Traveling by train, viewing all sorts of landscapes through the window.

Marker: Converts PDFs to Markdown using deep learning, supports all languages.

Intro to LLM’s – Slides from talk by Andrej Karpathy 

Recommended reading

Introducing Stable Audio Open – An Open Source Model for Audio Samples and Sound Design (stablityai)
Stable Audio Open is an open source text-to-audio model for generating up to 47 seconds of samples and sound effects. Users can create drum beats, instrument riffs, ambient sounds, foley and production elements. The model enables audio variations and style transfer of audio samples.

Mistral has released a software development kit (SDK), Mistral-Finetune, for fine-tuning its models on workstations, servers and small datacenter nodes.

The Impossibility of Fair LLMs We show that each framework either does not logically extend to LLMs or presents a notion of fairness that is intractable for LLMs, primarily due to the multitudes of populations affected, sensitive attributes, and use cases.

Recommended reading:

Mind as Machine by Margaret Boden (2006)
Weapons of Math Destruction by Cathy O’Neil (2016)
Algorithms of Oppression by Safiya Noble (2018)
Race after Technology by Ruha Benjamin (2019)
Discriminating Data by Wendy Chun (2021)
Artificial Knowing by Alison Adam (1998)
Computation and Human Experience by Philip Agre (1997)
The Closed World by Paul Edwards (1996)
Computer Power and Human Reason by Joseph Weizenbaum (1976)
Alchemy and Artificial Intelligence by Hubert Dreyfus (1965)
The Machinery Question by Maxine Berg (1980)

 

Capital

The special skill of each individual machine-operator, who has now been deprived of all significance, vanishes as an infinitesimal quantity in the face of the science, the gigantic natural forces, and the mass of the social labour embodied in the system of machinery, which, together with these three forces, constitutes the power of the master’.

Karl Marx, Capital, 1867

Mediaeater Digest Vol.30, No. 151

Announcing Showrunner  (The Simulation) We believe the future is a mix of game & movie. Simulations powering 1000s of Truman Shows populated by interactive AI characters. Welcome to Sim Francisco & Showrunner! With Showrunner, you’ll be able to WATCH and DIRECT stories from Simulations like Sim Francisco or Neo-Tokyo or Fallen London. Run out of episodes of your favorite AI show?  Generate another! Or let a simulation generate it for you based on its understanding of the characters.

Related : (HR Story – Hollywood Nightmare? New Streaming Service Lets Viewers Create Their Own Shows Using AI)

I could drag this out infinitely,   (Mario Klingemann ) but I think 9:24 is torture enough. New song “On and On”, celebrating  @suno_ai_ ‘s 3.5 model’s unrelenting will to keep the music going whatever it takes.

OpenAI finds Russian and Chinese groups used its tech for propaganda campaigns “We’ve seen them generating text at a higher volume and with fewer errors than these operations have traditionally managed,” Nimmo, who previously worked at Meta tracking influence operations, said in a briefing with reporters. Nimmo said it’s possible that other groups may still be using OpenAI’s tools without the company’s knowledge.

AI Integration and Modularization (Stratechery) Google – The first takeaway from this analysis is that Google’s strategy truly is unique: they are, as Nadella noted, the Apple of AI. The bigger question is if this matters: as I noted above, integration has proven to be a sustainable differentiation in (1) the consumer market, where the buyer is the user, and thus values the user experience benefits th

Utopian Realism, a speech by Bruce Sterling (From the Technology Biennial in Turin, Italy, April 02024.)  Five completely different self-invented unique toasters. Why? Why not just go buy the toaster at a store? Well — he very much wants to hand-make a toaster. He wants his toaster as a radically different toaster, the one that belongs to him. This is a “utopian device” in the sense of something that seems visionary, farfetched and silly. It’s just not practical, not realistic — but it’s practical and realistic for him. Calder tended to make art out of objects that the world had abandoned. Like the Turinese “Arte Povera” method — find junk, and dress it up, and re-format it. He had a different value system. To him this is is not junk. To him, this is a struggle for understanding

The Tech Baron Seeking to Purge San Francisco of “Blues” Still, his appetite for autocracy is bottomless. Last October, Balaji hosted the first-ever Network State Conference. Garry Tan—the current Y Combinator CEO who’s attempting to spearhead a political takeover of San Francisco—participated in an interview with Balaji and cast the effort as part of the Network State movement. Tan, who made headlines in January after tweeting “die slow motherfuckers” at local progressive politicians, frames his campaign as an experiment in “moderate” politics. But in a podcast interview one month before the conference, Balaji laid out a more disturbing and extreme vision.  (you can take the kid out of Long Island but not the LI out of the kid -ed ) 

-fast links

Curated directory of cybersecurity tools and resources 
RAG CLI
1-bit LLMs Could Solve AI’s Energy Demands “Imprecise” language models are smaller, speedier—and nearly as accurate

Mediaeater Digest Vol. 30, No. 149

Taxonomy of Human Rights Risks – Connected to Generative AI this document explores human rights risks stemming from the development, deployment, and use of generative AI technology. Establishing such a rights-based taxonomy is crucial for understanding how the United Nations Guiding Principles on Business and Human Rights (UNGPs) should be operationalised in addressing human rights risks connected to generative AI. This taxonomy is concerned with demonstrating how the most significant harms to people related to generative AI are in fact impacts on internationally agreed human rights.

Surveilling the Masses with Wi-Fi-Based Positioning Systems, we show that Apple’s WPS can be abused to create a privacy threat on a global scale. We present an attack that allows an unprivileged attacker to amass a worldwide snapshot of Wi-Fi BSSID geolocations in only a matter of days. Our attack makes few assumptions, merely exploiting the fact that there are relatively few dense regions of allocated MAC address space. Applying this technique over the course of a year, we learned the precise locations of over 2 billion BSSIDs around the world. (Location has become a victim of tech moving unchecked – wtf happend to Carpenter -ed)

Ransomware Group Claims Responsibility for Christie’s Hack At Christie’s, a spokesman said in a statement, “Our investigations determined there was unauthorized access by a third party to parts of Christie’s network.” The spokesman, Edward Lewine, said that the investigations “also determined that the group behind the incident took some limited amount of personal data relating to some of our clients.” He added, “There is no evidence that any financial or transactional records were compromised.”  (White glove opsec is not a thing here -ed)

OpenAI has announced that it is training its next generation of AI models, the follow-up to its state-of-the-art GPT-4 software which powers ChatGPT.   (what will be notable is what data is in and what data is opted out and what that looks like in product. -ed)

LeCun to Elon Musk on X about his latest AI goals- Join xAI if you can stand a boss who:
– claims that what you are working on will be solved next year (no pressure).
– claims that what you are working on will kill everyone and must be stopped or paused (yay, vacation for 6 months!).
– claims to want a “maximally rigorous pursuit of the truth” but spews crazy-ass conspiracy theories on his own social platform.

Ken Burns, H’24 Keynote Address to Brandeis University’s 2024 Graduates  “The checks of conscience are thrown aside and a deformed picture of the soul is revealed. The presumptive Republican nominee is the opioid of all opioids, an easy cure for what some believe is the solution to our myriad pains and problems. When in fact with him, you end up re-enslaved with an even bigger problem, a worse affliction and addiction, ‘a bigger delusion’, James Baldwin would say, the author and finisher of our national existence, our national suicide as Mr. Lincoln prophesies. Do not be seduced by easy equalization. There is nothing equal about this equation. We are at an existential crossroads in our political and civic lives. This is a choice that could not be clearer. (this is a text antidote to Harrison Butker’s BS -ed)

The Promise of Multi-Agent AI  Think of multi-agent systems as teams of experts, each contributing unique knowledge and abilities to collectively tackle difficult problems. Tasks are broken down into components and assigned to the agent best equipped to handle them. As each agent processes its part of the task and passes information to the next, the output is progressively refined and improved. Through such specialization, the resulting systems can achieve results that generalist agents struggle to match.

Steven Chu: ‘Wall Street analysts are totally amoral’ on climate FT In a lecture at the UK’s Royal Academy of Engineering, where he is an international fellow, Chu laid out the challenge. We sometimes refer to previous energy transitions — from wood to coal, or from coal to gas. Chu points out that they never really happened. Humans mostly added new sources of energy. Now, in less than three decades, we hope to transition away from oil, gas and coal. Chu doesn’t believe it’s possible, but he argues we must go as fast as we can.

The good news is the f