I recently got my hands on a GMKTec Evo X2 for local model inference. Here's my hardware details nish@gmktec-evo-x2:~$ sudo lshw -short H/W path Device Class Description ========================================================= system NucBox_EVO-X2 (EVO-X2-001) /0 bus GMKtec /0/0 memory 64KiB BIOS /0/b memory 1280KiB L1 cach

OpenAI's dev day was today. While I wrote up a short summary of what was announced on bluesky, one of the major announcements was the AppSDK for ChatGPT. It looks like OpenAI plans to position ChatGPT as a platform for the future not unlike the Google Play and

A Norm of a vector \(\vec{v}\) describes the magnitude or size of a vector. It is usually denoted as \(||\vec{v}||\). There are a few common norms worth discussing. Eucledian Norm (L2 Norm) Let's consider a vector v = [3, 4] we can calculate the Eucledean norm as,

A fantastic deep dive into the seahorse emoji phenomena[1] was recently published by Theia[2]. It's engaging, well presented and worth reading. The post presents a case using meta-llama/Llama-3.3-70B-Instruct. However I wanted to verify this behavior with smaller models which unsurprisingly fail the challenge as

Last week I happened to be in a discussion that involved getting an LLM to generate JSON reliably. A major frustration expressed was that no matter how much they tried the model would often fail to follow instructions during generation. I pointed out that most major vendors support some variant