• 0 Posts
  • 77 Comments
Joined 1 year ago
cake
Cake day: July 23rd, 2023

help-circle









  • I’ve been testing Ollama in Docker/WSL with the idea that if I like it I’ll eventually move my GPU into my home server and get an upgrade for my gaming pc. When you run a model it has to load the whole thing into VRAM. I use the 8gb models so it takes 20-40 seconds to load the model and then each response is really fast after that and the GPU hit is pretty small. After I think five minutes by default it will unload the model to free up VRAM.

    Basically this means that you either need to wait a bit for the model to warm up or you need to extend that timeout so that it stays warm longer. That means that I cannot really use my GPU for anything else while the LLM is loaded.

    I haven’t tracked power usage, but besides the VRAM requirements it doesn’t seem too intensive on resources, but maybe I just haven’t done anything complex enough yet.










  • The Asus Transformer Prime.

    It was an Android tablet circa 2011, right at that time they were actually making 10" Android tablets. I bought one as soon as Android Honeycomb launched which had an improved UI and lot of new tablet focused features. I bought the optional keyboard/battery attachment and planned for it to be my tiny laptop replacement that could also play emulators and be used for reading comics. I wanted to like it so bad.

    It never really panned out though, a large majority of which was because of the faulty Nvidia Tegra 3 chip. Awful performance issues, terrible wireless connectivity, overheating, battery drain and nonexistant software updates from Asus. I ran custom ROMs trying to squeeze it as much as I could but that meant I was constantly tinkering with it and having yet more problems. Eventually I even broke the screen (my fault) and painstakingly went through a whole botched screen replacement before finally deciding that it had been a huge waste of time and money and sent it to it’s grave.