Neural Processing Units (NPUs)

Topic

New Reply

I’m shopping for a new Dell laptop. Some models come with an Intel “Core Ultra processor” which includes a “neural processing unit (NPU)” in addition to the CPU and GPU. Dell is marketing the NPU as “built-in AI.” The NPU is a separate chip. What would an NPU do and why would I want one (or not)? At this point, I am leery of AI in general and Microsoft Copilot in particular. Can the NPU function on its own or does it require Copilot? There are plenty of Dell laptops without the NPU. I’m just curious about its function.

My knee-jerk reaction to all AI at this point is that adding AI to anything is meant to enhance the AI itself, not the user experience. Planting AI on thousands (or hundreds of thousands, or millions) of PCs will allow the AI an unprecedented opportunity for machine learning at the expense of users. Anything deemed worthwhile by the AI or its master would be phoned home. I cannot understand the use-case for AI on a home computer.

Will appreciate any comments clarifying this technology and its purpose. I plan to avoid it. Change my mind.

5 users thanked author for this post.

PL1, Cybertooth, windbg, Charlie, NaNoNyMouse

Reply | Quote

Replies

Microsoft’s own description of “Neural Processing Units” and its “Copilot+” technology is here:

https://www.microsoft.com/en-us/windows/copilot-plus-pcs

It’s pretty terrifying stuff from what I’ve skim-read of it – essentially handing more control from the user of the device to the device itself and, crucially, recording screenshots every few seconds.

The UK’s data watchdog (the ICO) has requested more info about Copilot+ from MS:

https://www.bbc.co.uk/news/articles/cpwwqp6nx14o

1 user thanked author for this post.

NaNoNyMouse

Reply | Quote

What I’ve learned from Google so far:

1. Intel is changing the naming system of its Core processors. The premium tier will be called “Core Ultra,” which will include AI-powered capabilities.

2. The “Recall” feature (the one that records screenshots every few seconds) is limited to Win11 with the “Copilot+” enabled. Copilot+, in turn, requires an NPU with performance rated at 40 trillion operations per second (TOPS). That performance metric is currently met by only one chip soon to enter the market, Qualcomm’s Snapdragon X. To use Recall, users must purchase a Copilot+ PC powered by Snapdragon X with neural processing unit (NPU) included. So none of the current Dell “built-in AI” computers will run Copilot+ or Recall.

3. With respect to privacy issues surrounding Recall, Microsoft says,

“[T]he Recall index remains local and private on-device, encrypted in a way that is linked to a particular user’s account. Recall screenshots are only linked to a specific user profile and Recall does not share them with other users, make them available for Microsoft to view, or use them for targeting advertisements. Screenshots are only available to the person whose profile was used to sign in to the device.”

As with nearly all technology vendors, if Microsoft’s lips are moving, they are lying. There is no telling how Recall might be abused in the future. I plan to stay the hell away from it.

4. Insofar as NPUs in general, the information I have been able to find is couched in meaningless (to me) technobabble like this:

“An NPU, or Neural Processing Unit, is a dedicated processor or processing unit on a larger SoC designed specifically for accelerating neural network operations and AI tasks. Unlike general-purpose CPUs and GPUs, NPUs are optimized for a data-driven parallel computing, making them highly efficient at processing massive multimedia data like videos and images and processing data for neural networks. They are particularly adept at handling AI-related tasks, such as speech recognition, background blurring in video calls, and photo or video editing processes like object detection.”

It remains unclear to me whether the NPU contains embedded AI features, or, in the absence of AI installed on the local machine, resembles a CPU without an operating system installed.

Since I intend to be a laggard in adopting AI, none of this matters to me except as needed to identify hardware and software I do not want to buy, or to disable any AI features embedded in Win11.

4 users thanked author for this post.

PL1, Cybertooth, windbg, Charlie

Reply | Quote

WUBRINY wrote:

4. Insofar as NPUs in general, the information I have been able to find is couched in meaningless (to me) technobabble like this: “An NPU, or Neural Processing Unit, is a dedicated processor or processing unit on a larger SoC designed specifically for accelerating neural network operations and AI tasks. …

I added the bolding in the quote.

To help clear things up a bit for you, when written like it is above while being used in a computer sector-related article, “SoC” usually means “System on a Chip”. Details of this concept are at the following location on the Wikipedia site:

https://en.wikipedia.org/wiki/System_on_a_chip

I hope this helps a bit!

Reply | Quote

I get the system on a chip concept, but is SoC capable of including embedded AI? I was not able to discern the answer to that question from the Wikipedia article.

I have resolved the immediate problem by purchasing a laptop with an Intel Core i7 CPU rather than a Core Ultra.

Reply | Quote

SoC can include anything the designer wants, including AI processing. The idea is to limit additional components so you can reduce both size and cost.

cheers, Paul

Reply | Quote

Skynet will take over the world using NPUs on SoC chips with embedded AI.

1 user thanked author for this post.

Cybertooth

Reply | Quote

Came across an editorial at Ars Technica today titled “Windows Recall demands an extraordinary level of trust that Microsoft hasn’t earned.” The title gives the gist of the editorial. There is some discussion of the technical aspects of Recall. Here are a few relevant quotes:

* [A]dmin access to the system isn’t required to read another user’s Recall database. Another user with an admin account can easily grab any other user’s Recall database and all the Recall screenshots by clicking through a simple UAC prompt. The SQLite database is stored in plain text, and data in transit isn’t encrypted, either, making it trivially easy to access both the stored database of past activity and to monitor new entries as Recall makes them. Screenshots are stored without a file extension, but they’re regular old image files that can easily be opened and viewed in any web browser or image editor.

*  ecause Recall is on by default and you have to manually exclude specific apps or websites from being scraped by it, the SQLite database will keep records of activities that are explicitly meant to be hidden or temporary. That includes viewing pages in Incognito mode in some browsers, emails or messages that you delete from your device, and files that you edit or delete.

* [G]iven the sheer amount of data that Recall scrapes, the minimal safeguards Microsoft has put in place to protect that database once a malicious user has access to your PC, and the fact that many PC users never touch the default settings, the risks to user data seem far higher than the potential benefits of this feature.

Same old Microsoft shenanigans.

1 user thanked author for this post.

Cybertooth

Reply | Quote