• Welcome to TechPowerUp Forums, Guest! Please check out our forum guidelines for info related to our community.
  • The forums have been upgraded with support for dark mode. By default it will follow the setting on your system/browser. You may override it by scrolling to the end of the page and clicking the gears icon.

AMD Publishes User Guide for LM Studio - a Local AI Chatbot

T0@st

T0@st

News Editor
Joined
Mar 7, 2023
Messages
3,247 (3.98/day)
Location
South East, UK
System Specs
System Name The TPU Typewriter
Processor AMD Ryzen 5 5600 (non-X)
Motherboard GIGABYTE B550M DS3H Micro ATX
Cooling DeepCool AS500
Memory Kingston Fury Renegade RGB 32 GB (2 x 16 GB) DDR4-3600 CL16
Video Card(s) PowerColor Radeon RX 7800 XT 16 GB Hellhound OC
Storage Samsung 980 Pro 1 TB M.2-2280 PCIe 4.0 X4 NVME SSD
Display(s) Lenovo Legion Y27q-20 27" QHD IPS monitor
Case GameMax Spark M-ATX (re-badged Jonsbo D30)
Audio Device(s) FiiO K7 Desktop DAC/Amp + Philips Fidelio X3 headphones, or ARTTI T10 Planar IEMs
Power Supply ADATA XPG CORE Reactor 650 W 80+ Gold ATX
Mouse Roccat Kone Pro Air
Keyboard Cooler Master MasterKeys Pro L
Software Windows 10 64-bit Home Edition
AMD has caught up with NVIDIA and Intel in the race to get a locally run AI chatbot up and running on its respective hardware. Team Red's community hub welcomed a new blog entry on Wednesday—AI staffers published a handy "How to run a Large Language Model (LLM) on your AMD Ryzen AI PC or Radeon Graphics Card" step-by-step guide. They recommend that interested parties are best served by downloading the correct version of LM Studio. Their CPU-bound Windows variant—designed for higher-end Phoenix and Hawk Point chips—compatible Ryzen AI PCs can deploy instances of a GPT based LLM-powered AI chatbot. The LM Studio ROCm technical preview functions similarly, but is reliant on Radeon RX 7000 graphics card ownership. Supported GPU targets include: gfx1100, gfx1101 and gfx1102.

AMD believes that: "AI assistants are quickly becoming essential resources to help increase productivity, efficiency or even brainstorm for ideas." Their blog also puts a spotlight on LM Studio's offline functionality: "Not only does the local AI chatbot on your machine not require an internet connection—but your conversations stay on your local machine." The six-step guide invites curious members to experiment with a handful of large language models—most notably Mistral 7b and LLAMA v2 7b. They thoroughly recommend that you select options with "Q4 K M" (AKA 4-bit quantization). You can learn about spooling up "your very own AI chatbot" here.



View at TechPowerUp Main Site | Source
 
AMD believes that: "AI assistants are quickly becoming essential resources to help increase productivity, efficiency or even brainstorm for ideas."

Yeah nah.
 
This is really such an insane f**king joke. They've "caught up" by just publicly admitting how incompatible all their products are with LLMs?
 
Minus Infinity said:
AMD believes that: "AI assistants are quickly becoming essential resources to help increase productivity, efficiency or even brainstorm for ideas."

Yeah nah.
Click to expand...
It can be useful for research, provided you curate the data it's searching well. It's a pretty select use case and it is certainly overblown, but even our sites founder, W1zzard, has admitted this is a handy trait.

tl;dr: Is AI overhyped? Yes. Is it useless? No.
 
R-T-B said:
It can be useful for research, provided you curate the data it's searching well. It's a pretty select use case and it is certainly overblown, but even our sites founder, W1zzard, has admitted this is a handy trait.

tl;dr: Is AI overhyped? Yes. Is it useless? No.
Click to expand...
In the context of a home PC it largely is useless. I know full well how powerful AI can be in science, engineering and medicine to name a few, but you can bet they aren't relying on a BS copilot.
 
Minus Infinity said:
In the context of a home PC it largely is useless. I know full well how powerful AI can be in science, engineering and medicine to name a few, but you can bet they aren't relying on a BS copilot.
Click to expand...

Got to keep hyping it whilst some killer non niche use case is found for LLMs.
 
Minus Infinity said:
In the context of a home PC it largely is useless. I know full well how powerful AI can be in science, engineering and medicine to name a few, but you can bet they aren't relying on a BS copilot.
Click to expand...

It can still be usefull to bounce simple ideas from, it's useless in the context companies like Microsoft want to push it, integrating it in all corners of the OS. But having a chat bot quickly accessible and offline is pretty cool not to mention because the free party we've had with chatgpt and bard/gemini will certainly end in the not so distant future as it's simply not a sustainable service, openAI is literally bleeding money as is google, though certainly much much less because they're less used and they were much better prepared with accelerators built in house ready to go.
 
Minus Infinity said:
In the context of a home PC it largely is useless. I know full well how powerful AI can be in science, engineering and medicine to name a few, but you can bet they aren't relying on a BS copilot.
Click to expand...
That's a fair assesment.
 
AMD, always chasing after NVIDIA with subpar implementations of the cool things that NVIDIA does, without regard for the obvious elephant in the room which they absolutely refuse to even acknowledge, let alone do anything about it.

For an answer to "Chat with RTX", that even someone with zero knowledge of computers could use, this is sorely lacking.
 
Funnily enough, their guide didn't work for me. I couldn't get any models to run on my 7900 XTX. But it did point me in the right direction which was KoboldCPP: https://github.com/YellowRoseCx/koboldcpp-rocm

This was VERY easy to set up and get running and it has ROCM support in Windows too.
 
hilarious. at the same time one can ollama on linux on multi (mixed) GPU automagically starting with rocm 6.0 release since december 2023. i'm running it on 780m igpu + 7600m xt egpu (via usb4).
 
Yeah it's been a difficult time for me with my RX 7800 XT. Feel now I can only use it for Gaming because of no official ROCm support. Only RX 7900 is getting goodies. I hope LM Studio succeeds. They are the only ones trying to get this to work on AMD. I'd rather not talk about DirectML.
 
ShaneTeks said:
Yeah it's been a difficult time for me with my RX 7800 XT. Feel now I can only use it for Gaming because of no official ROCm support. Only RX 7900 is getting goodies. I hope LM Studio succeeds. They are the only ones trying to get this to work on AMD. I'd rather not talk about DirectML.
Click to expand...
wrong. it works great under linux. both images (sd.next) and LLMs (ollama)
 
rhqq said:
wrong. it works great under linux. both images (sd.next) and LLMs (ollama)
Click to expand...

Well, this has always been the problem with AMD GPUs, hasn't it? "It works, if you install A, configure B, set environment for C, compile D, adjust E, and make sure to use Linux because otherwise you ain't getting anything done."
 
Dr. Dro said:
Well, this has always been the problem with AMD GPUs, hasn't it? "It works, if you install A, configure B, set environment for C, compile D, adjust E, and make sure to use Linux because otherwise you ain't getting anything done."
Click to expand...
it just shows how little you know about the thing. the whole market for AI, both NV and AMD is almost purely Linux. things just work there, unlike windows.

so again: it works great, if you follow the industry standard.
 
rhqq said:
wrong. it works great under linux. both images (sd.next) and LLMs (ollama)
Click to expand...
Have you perhaps got ComfyUI to work with the above gfx1100, 1101, 1102? and have you tested and compared the performance running SD with DirectML compared to using ROCm?
 
You must log in or register to reply here.
Back
Top