DeepSeek and the AI Bubble; Napkin Calculation Part 2

We continue from DeepSeek and the AI Bubble; Napkin Calculation. You may wish to first peruse AI for Dummies, in Bite-Size Pieces, Part 1

Some may accept the probabilities outlined in DeepSeek and the AI Bubble; Napkin Calculation  as worthy of consideration in the same manner as the Technological Singularity  Both are typical of futurology,  lacking hard justification. Yet those who have infrequent contact with futurology may demand a little more justification of the napkin calculation. The gist is this: When there exist a sufficient number of reasonably independent ways an event can happen, that event has a statistical character. Mathematicians use this very precisely with the ergodic theorems; we take license, applying it imprecisely but with strong analogy.

AI is a field where many things can happen. See (Wikipedia) Timeline of artificial intelligence. 1943 saw publication by Warren Sturgis McCulloch and Walter Pitts of the seminal paper, “A Logical Calculus of the Ideas Immanent in Nervous Activity.” Since then, many speculative AI ideas have temporarily been the focus of attention, with minor development of some, before sinking under the intermittent surge of newer ones.

In most fields, the past is littered with discredited ideas. Rather uniquely, AI is not. The vast majority of the undeveloped past of AI remains fertile ground. Most of it is eligible for renewed interest. We cannot tell which idea and when; the statistical principle applies.

The ideas of AI bedrock, which are distinct from modeling, are a set of contradictions, governed by tradeoffs and practicality:

  • Structure versus amorphous hardware architecture.
  • Self organization of the hardware versus design.
  • Self organization of the weights versus specification.
  • While training, algorithmic versus statistical operation.
  • Neuromorphic versus simulation of neuronal structures.
  • Strong AI versus Weak AI.

The first  item is relevant to DeepSeek, so let’s first focus on it.  The designers  claim to obtain performance comparable ChatGPT4 while using less: energy, computing resources, and training. Detractors suspect that DeepSeek cleverly  cannibalized other AI’s, with the premasticated data responsible for the training economies. Even if this turns out to be the case, a significant aspect of the DeepSeek claim may be due to genuine improvement.

In the early years of AI,  notwithstanding the early work of McCullough and Pitts, neural networks were assumed to have little in the way of complex structure — other than that induced by the weights. In theory, if every neuron were connected to every other neuron, with a structure that resembles an amorphous glass in hyperspace, if you knew how to set the weights or train the thing, the useless/redundant connections would vanish, and the optimal network would  self-assemble. This has no practicality, with time to train unrealizable in the real world. It does  have a mysterious attraction, the possibility of elusive, spontaneous, general intelligence, — g.

So structure was imposed. Some structures, like  Rosenblatt’s perceptron, are layered, with an orderly progression from input to output. Some structures incorporate loops, saving processing units by re-use which allow the net to consider the results of its work multiple times. Some networks start with everything zeroed out, while others are initialized with hopefully helpful patterns. With the advent of large language models, layered  systems took the lead.

Why they took the lead is not explainable in formal terms, because neural nets are the Wild West of mathematics. There appears to be a lot of truth in this bold statement: If  a principle can be proven, it has limited utility, and if it is useful, it can’t be proven. The atmosphere contrasts with the frontiers of physics,  where the latest developments are rigorous and disconnected from validation.

The hardware support for a natural language model has to be sufficient  to store enough information to discriminate between all the patterns – words, sentences, logical statements,  etc., that the designers intend to train with. Given a vocabulary of a certain size, how big does a network have to be? The Wild West has no answer to this question. The designers of ChatGPT et  al. proceed  empirically. If  a designer desired  a better answer, he would look at two papers: On Neural Network Kernels and the Storage Capacity Problem and Memory capacity of large structured neural networks. You need look only at the abstracts. The first paper works  with a two layer, infinite network, which cannot be made or used, to derive some rigorous results that have no real world application. The second abstract depicts frustrated, limited research on more practical configurations, but is noteworthy for the ratio of kvetch to result.

In consequence, there is no first-principles way to tell how efficient a net is at storage. Try it and see is the order of the day. Regardless of the truth or lack, of accusations that DeepSeek is a DeepFake, this uncertainty is one hole DeepSeek  found, crawled through, and successfully exploited.

This is a lot for your neural network to digest, so we’ll continue shortly.

 

 

 

 

 

 

 

 

DeepSeek and the AI Bubble; Napkin Calculation

The claims of DeepSeek are highly credible, and should strike fear into the heart of anyone playing this bubble. AI Winter follows AI Summer. A serial “kill-chain” of risks can now be described, of which the failure of any one will lead to DeepFreeze of DeepWinter. The serial possibilities:

  • Insolvency of investments in generation infrastructure.
  • The  U.S. is no longer the top player.
  • Projected benefits fail to materialize.
  • Labor unrest, spreading as disenfranchisement proceeds apace. Luddites are inspired to make the last stand of humanity.
  • A distinct category of unrest, caused by the use of AI by the unscrupulous to manipulate the information space.

The failure of any one item in the above list punctures the current projections. The multiplicative uncertainty of multiple factors may be sufficient.

We assume that frontier engines such as ChatGPT4 approximate human capabilities. In fact, they are clearly superior, in many categories of formal reasoning, to the average human. Rumor has it they may shortly exceed superior human intellects.

We seek to answer, in a napkin calc, whether the claims of DeepSeek are plausible.

  • A center value estimate for the amount of electricity used to train ChatGPT4 is 5.7 * 1010 watt-hours.
  • There are 8760 hours in a year.
  • The brain develops anatomically till maturity at age 25, a space of 219,000 hours.
  • The brain consumes about 25 watts of power = 25 watt-hours per hour.
  • Over 25 years, the brain consumes roughly 5.47 * 106 watt-hours of energy.
  • The ratio of total training energy consumption, ChatGPT4 ÷ human = 5.7 * 1010 ÷5.47 * 106, which is approximately

10,000 : 1

The energy reduction of DeepSeek is made plausible by what is possible.

  • This ratio is approximately 213.
  • The time remaining until the Technological Singularity, posited by some for 2038, is 13 years. This implies an annual halving of power consumption per AI.

Anyone with skin in the game who has given this ratio consideration has sleepless nights worrying about being blind-sided, which is exactly what happened. Group-think takes care of the rest. Ironically, human nature suggests if the ratio of energy consumption were more like 100:1, it would get our attention. But for the quants who “know everything”, the number does not compute. Incomensurable numbers do not exist. Hence, it does not exist.

So why not copy the brain? Do we need to? What is the prospect for further disturbing surprises? Due to time constraints , this will be continued shortly.

 

 

(Al Jazeera) Russia tight-lipped on Syrian demand of al-Assad for military bases

(Al Jazeera) Russia tight-lipped on Syrian demand of al-Assad for military bases. Quoting,

Russia has declined to comment on reports that Syria has demanded the return of Bashar al-Assad in return for allowing Moscow to maintain its military bases in the Middle Eastern country.

There is a traditional face-saving solution: Assad assassinated in Moscow by radicals recruited by the KGB or GRU from central Asia.  The assassins never know who they are actually working for. Following the accomplishment, the assassins themselves are liquidated.

Similar action could be facilitated with Assad first receiving asylum in northern Cyprus, where the survival record of Russian traitors is notably poor.

An alternative is the “unfortunate accident”, such as a fall down the stairs, or defenestration, but this requires coordination with Assad’s security detail.

And, of course, there is always Novichok.

 

CNN Shuffles the Lineup; Front Page Yellow Snow

(The Independent)  CNN staffers blast ‘tone deaf’ CEO Mark Thomp son over layoffs as  Jim Acosta mulls move to midnight slot.

About the layoffs, Thompson appears at a loss for words for mere  personal tragedy. We have more consequential fish to fry than his vocabulary. Quoting,

…revealed all of the changes to the channel’s weekday lineup. Besides the new Blitzer and Brown program, which will air for two hours daily beginning at 10 a.m. ET, CNN This Morning will now be a one-hour show hosted by Cornish and air at 6 a.m. ET. Kasie Hunt, who currently helms CNN This Morning, is moving to afternoons to host The Arena…

…Jake Tapper’s The Lead will shift later in the day and air from 5 p.m. to 7 p.m. ET. …Rahel Solomon will kick off the network’s weekday morning slate with the 5 a.m. show 5 Things… Kaitlan Collins’ primetime show is also now shifting primarily to Washington, D.C…

They are all extraordinary talent, but this shuffle will actually accelerate the decline. Quoting from Att: David Zaslav; Future of CNN,

…The effect on CNN has been strongly negative, intimately related to the transition of cable news to the phase of decline:

    • Demand for novelty is absent.
    • Retention is based on loyalty.
    • Loyalty abhors change.
    • In consequence, any change Licht could or did make resulted in viewer loss. The transition from maturity to decline is a programming  “game changer.” It rendered Licht’s  prior experience useless.

Quoting The Independent,

…critics wondered if burying Acosta – who the president despises – in late-night was an effort to curry favor with the new administration…Others…weren’t entirely sold…. “Honestly, this place is such a mess,” one CNN on-air personality told The Independent. “I’d actually be relieved if ANY decisions had a rationale behind them. Even one like that.”

Quoting from Intel9.us Up After a 4 Day Outage; Note to CNN & David Zaslav,

CNN, before you upend the lives of hundreds of faithful employees, you should seek both internal and EXTERNAL advice. Positive change can come fast: Though Rome wasn’t built in a day, Mark Zuckerberg wrote the original Facebook in a month.

This sketches the problem: Att David Zaslav; Warner Bros. Discovery signals rapid deterioration of television business, sending stock plummeting.   My telephone hasn’t noted any outreach, so here’s a  freebee. In the space of three weeks, CNN ran three front page stories about pee. Yeah, that stuff. The word actually appeared in the titles. They were placed in the right hand column, intended anodyne to the misery of the left hand column. I felt pissed off. All the years I had faithfully followed the advice, “Don’t eat yellow snow”,, and now this!

After their front page runs, the three P’s had a kind of afterlife, slapped into different layouts with  differing logic, until finally melting away with a swirling gurgle down the drain, a box labeled “search.”

I have lots of ideas but I shall restrict myself here to one bit that doesn’t require elaboration:

***Don’t run yellow snow.***

 

 

 

 

 

 

(CNN) Israeli foreign minister: Trump admin. didn’t force Israel into accepting ceasefire ; Future Trends of M.E. Terrorism

(CNN) Israeli foreign minister: Trump admin. didn’t force Israel into accepting ceasefire.

The strategies of Hezbollah and Hamas have been similar, with

  • Quasi state attributes.
  • Broad application of terror.
  • Goals of conflict exceeding terror, at levels of conventional war necessary to destroy a state.

While Hezbollah prepared meticulously for undated future action, Hamas took bold, précipité action, intending to establish a new reality,  perhaps  to drag other belligerents into active conflict.

When the early predictions of pundits were that Israeli intervention into Gaza would be surgically informed by precise intelligence, I knew the opposite. I chose not to share this because, despite whatever misgivings I may have, I am not an impartial observer. Rather than mislead, I simply chose not to write about it at all.

The near total physical destruction of the infrastructure of Gaza was mandated by the inability of Israeli forces to successfully implement a surgical approach with these goals:

  • Combat losses minimized to a level sustainable for the duration of the conflict.
  • Sufficiently complete destruction  of Hamas so as to impede regeneration.
  • Creation of a political vacuum in which a new civil authority can be created.
  • Destruction of the myth that with enough stockpiling, digging, training, and waiting, the Intifada could achieve asymmetric parity with Israel.

What remains of political Hamas cannot spin the result of this conflict as a conditional or partial victory. In consequence, a certain percentage of Gazans, an as yet unknown but very important number, will resist the attempts of Hamas to resume.  Whether this creates a political vacuum remains to be seen.

Hezbollah was seen as the far more formidable adversary. Their undoing began as a technological masterstroke; see Israel’s Sabotage Coup; How to Make an Exploding Pager.  While the highly acceptable result has not been so nearly complete as with Hamas, the loss to Iran of Syria as a support land bridge has amplified the achievement.

These twin results, as pleasant as they may be to contemplate in the short term, are seeds of future conflict, abetted by the cultural myths and memes prevalent in the region. Quoting from In the New Year; The False God of Expertise; What to do on New Years Eve?,

Histories of mankind have various perspectives: politics, culture, technology,  and conflict.  But one slant of human history is so concisely describable, and so negative, it has been neglected by historians as the driver of so much: “Destruction of life and wealth in the service of mythology.”

The victors of conflict tend toward the error of permanence, viewing shattered dreams as shattered glass. Societies driven by mythology behave more like distributed systems, carrying with incredible resilience a cultural and mythological payload. Everyone who has seen a particular Hollywood movie has received a quick introduction to distributed systems, courtesy of a former Governor of California: (YouTube) Come with me if you want to live.

The first resource of  prediction is extrapolation of the current trend, which may appear to be a lull.  It denies the dynamic nature of adversaries, which are in the early stages of

  • post conflict assessment.
  • political reorganization.
  • accumulation of resources.
  • revision of strategy.
  • resumption of conflict.

The state actor. Iran’s attempt at empire has left it with an excess of management structure. Previously occupied with administering elements in  Lebanon, Syria, and Iraq, only Iraq is left. One doesn’t waste that kind of talent, so it is likely to be put to work in Iraq, elevating near-term risk to that country.

Both sides of this conflict incorporate religious/mythological elements, though the proportions differ. The West, and Israel still, are reliant, post WW2, on a modern legacy of secular thought. This manifests in our astonishment of Russian casualties in Ukraine in a totally optional war. It was preceded in the Iran-Iraq war by Iran’s use of child martyrs; see Child Soldiers in Iran.

Non-state-actors. Our continual astonishment at barbarities could result in lack of consideration of the more dire modalities should this conflict resume.  The actions of non-state actors seem to defy the restraint of any form of logic. The failure of the intifada will cause these actors to explore more extreme strategies, without apparent limit to self-harm.

One of our secular givens is that these terror groups are basically fighting for a life on earth. This implies that the land of Israel must be regained as a habitable place. This negates the possibility of a completely religious motivation, the land now a mere symbol of possession, regained as a completely sterile wasteland, incapable of human habitation. The reservoir is Syria; see After Assad; a Terror Nation-State.

This implies that non-state actors will attempt the use of weapons of mass destruction without regard for the populations for which they presumably advocate. The use of Gazans as human shields is an early form of this meme, which will evolve with greater virulence.

There is no off-switch:

***Come with me if you want to live.***

 

 

 

 

 

 

LA Fires; Reality of the Anthropocene Epoch; Rise of the Island Refuge

The media concentrates on the human tragedy of the here and now. The big,  obvious, unspoken fact  is permanent change. There can be no going back to  the past of only a few days prior, when LA residents  lived in stressed disharmony with nature. In LA, the hostility of the planet manifests in all four of the ancient elements: earth, air, fire, and water. Far below lie earthquake faults;  on the surface,  red shale cliffs formed of incompletely metamorphosed clay and vulnerable to water; above, wind and fire,  preceded by atmospheric rivers.

No one  is calling   this a hundred year event; it’s the new normal. Some migration will occur, slowed by the national shortage of housing stock.  Some rebuilding will be attempted, with houses constructed of nonflammable materials, such as concrete,cinder block, and steel or tile roofs. Homeowners will still have to evacuate.

This doesn’t answer the bigger  question: How can the core city be protected from mass conflagration exceeding the great urban fires of the 19th century? If the current event were juxtaposed with a major earthquake that disrupted the water supply, millions could die.

Los Angeles is the only city in the world — in the broad sense of Los Angeles County — divided by a mountain range, the Santa Monica Mountains.  Vegetation is required to hold the land in place, land where people have chosen to build.  Vegetation burns. Walking along the unpaved portion of Muholland Drive in early fall, one is struck at how simple a thing as a dropped match could cause a conflagration. In an environment  where vigilance is required of everyone, all the time, bad things happen. The Griffith Peak blaze was caused by a sleepy bum with a cigarette.

We can take a lesson from the Mound Builders of pre-Columbian   America.  Although most mounds appear to have been built for cultural reasons, some of the earliest seem to have been  artificial islands, refuges from Mississippi floods. Los Angeles needs islands of habitability,  walkable sub-cities, constructed of nonflammable materials, with carefully minimal vegetation.

The requisite size of these mini-Manhattans is determined by their function as fire breaks, with breadth greater than an ember can travel. They don’t have to be ugly. Embrace nature at the risk of your life, or distance, living safely in hopefully attractive artifice. One of these is manifest disharmony, the other tolerable.

If this sounds like an unpleasant dream, you won’t have to wait a hundred  years for a repetition. Fire season beckons. There will be a natural demand for fire-proof communities.

You can still experience  the striking natural beauty of LA. You just can’t nestle  in its embrace.

***Koyaanisqatsi***

 

 

 

 

(NBC) Rare explosive found in New Orleans attack; The Bomb in Your Coffee

(NBC) Rare explosive found in New Orleans attack. Quoting,

Federal investigators examining the ISIS-inspired terror attack that killed 14 in New Orleans report that Jabbar used a very rare explosive compound in the two functional IEDs placed prior to the vehicle-ramming attack…

Start here: Forensic analytical aspects of homemade explosives containing grocery powders and hydrogen peroxide.  Quoting,

Homemade explosives become a significant challenge for forensic scientists and investigators. In addition to well-known materials such as … perpetrators often produce more exotic and less recognized Homemade Explosives (HMEs). … Interestingly, similar explosive properties are found in mixtures of numerous solid materials with H2O2. Notably, powdered groceries, such as coffee, tea, grounded spices, and flour, are particularly interesting…

Coffee appears to be the most powerful. Start your day with a bang. How would the New Orleans bomber become familiar with the chemistry?  A secret cabal is unnecessary. Quoting,

Non-professional pyrotechnics and chemistry enthusiasts form a distinct group of HME producers. Modern technologies provide these individuals with unlimited access to diverse sources, such as scientific papers, books, manuals, patents, government reports, web tutorials, and discussion boards, contributing to the growing diversity and complexity of HMEs and IEDs.

Some readers may associate H202 (hydrogen peroxide) with the notoriously unstable TATP.  But it can also be used to produce relatively stable and powerful explosives from uncontrolled substances available at grocery stores.

I’ll take mine with cream.

New Orleans & Las Vegas Bombings; Saved by Mediocrity; Propaganda of the Deed

Although these bombings might appear to incorporate elements of sophistication, they manifest deficiencies:

  • Lack of technical expertise in manufacturing IEDs.

  • Deficient planning skills.

  • Limited resources of lone wolf operators.

For obvious reasons, I will not elaborate. Even as failures relative to grand ambitions, these events provide inspiration to would-be terrorists, some of which will have more ability.

Since the advent of social media, we have seen the legitimization of violence in cryptic subcultures. The trend continues to expand into other cultures, now as an alternative to political process. Lone wolves will expand into cells.

The precedent is 19th century anarchism. Within the last month alone, we have seen three instances of propaganda of the deed.

The estimate is that, powered by social media, this trend will continue to escalate, gaining resources, frequency and effect.