There’s a solution to abuse the Hugging Face Safetensors conversion instrument to hijack AI fashions and mount provide chain assaults.
That is in line with safety researchers from HiddenLayer, who found the flaw and revealed their findings final week, The Hacker Information reviews.
For the uninitiated, Hugging Face is a collaboration platform the place software program builders can host and collaborate on limitless pre-trained machine studying fashions, datasets, and functions.
Altering a extensively used mannequin
Safetensors is Hugging Face’s format for securely storing tensors which additionally permits customers to transform PyTorch fashions to Safetensor by way of a pull request.
And that’s the place the difficulty lies, as HiddenLayer says the conversion service could be compromised: “It is attainable to ship malicious pull requests with attacker-controlled knowledge from the Hugging Face service to any repository on the platform, in addition to hijack any fashions which might be submitted by the conversion service.”
So, the hijacked mannequin that’s presupposed to be transformed permits menace actors to make adjustments to any Hugging Face repository, claiming to be the conversion bot.
Moreover, hackers may also exfiltrate SFConversionbot tokens – belonging to the bot that makes the pull requests – and promote malicious pull requests themselves.
Are you a professional? Subscribe to our e-newsletter
Signal as much as the TechRadar Professional e-newsletter to get all the highest information, opinion, options and steering your corporation must succeed!
Consequently, they may modify the mannequin and arrange neural backdoors, which is basically a complicated provide chain assault.
“An attacker might run any arbitrary code any time somebody tried to transform their mannequin,” the analysis states. “With none indication to the person themselves, their fashions may very well be hijacked upon conversion.”
Lastly, when a person tries to transform a repository, the assault might result in their Hugging Face token getting stolen, granting the attackers entry to restricted inner fashions and datasets. From there on, they may compromise them in numerous methods, together with dataset poisoning.
In a single hypothetical state of affairs, a person submits a conversion request for a public repository, unknowingly altering a extensively used mannequin, leading to a harmful provide chain assault.
“Regardless of the perfect intentions to safe machine studying fashions within the Hugging Face ecosystem, the conversion service has confirmed to be weak and has had the potential to trigger a widespread provide chain assault by way of the Hugging Face official service,” the researchers concluded.
“An attacker might acquire a foothold into the container operating the service and compromise any mannequin transformed by the service.”
<header
