Guten Tag!

Many greetings from Munich, Germany. In today’s newsletter I want to teach you how you can easily send your data to local LLMs, i.e. Large Language Models (you know, the cool AI stuff). This can be useful for all kinds of text processing. And to do so, we’re going to take a look at the brand-new {mall} package. Let’s dive in.

Install the {mall} package

Let’s install the {mall} package. Nothing easier than that.

Install ollama

But wait, there is more. All LLMs will be served locally via ollama. “Serve locally, you say? Never heard of that!”

Well, in that case, let me explain. All LLMs like ChatGPT are basically a very large neural network. Sounds biological but can in fact be represented by a VEEEERY large matrix of so called weights (technically it’s more than just one single matrix but I’m oversimplifying here for brevity so bear with me.)

In any case, these weights will be used to determine what kind of response the LLM produces. So, if you have that matrix, you can run your own ChatGPT. It’s kind of the key to doing hot AI stuff.

Of course, openAI keeps its matrix under wraps but there are other open-source folks who say: “Here, stranger, take my matrix and run your own calculations with that.” Cool beans! But we hate doing calculations manually, don’t we? That’s where ollama comes in. 💡 

It’s a tool that can run on your computer and download matrices from open-source vendors. So if you want to send a prompt to an LLM, then you (or your R code) will have to knock at ollama’s door (which is running in the background) and say “Hey, run this prompt against <model-x>” And then ollama is hopefully nice and says: “Lucky you, I’ve got this model in store for you so I’m serving you that juicy dish LLM.”

Okay, so hopefully this dumb little explainer demystified this stuff about “serving LLMs” a little bit. With that said, head over to ollama and install it for your operating system.

Get a local model through {ollamar}

Nicey, nicey! We have {mall} installed for R and we have ollama installed on our computer. Time to get an LLM. That’s where the {ollamar} package comes in. It’s one of the packages that is also installed when you install {mall}. And it handles the interaction between R and ollama.

Inside of R, we can run the pull() command to download an LLM (you know, the matrix that represents the fancy neural network). You can find a whole list of LLMs that ollama can download for you on their website. Here, we’re just going to use a tiny model called “llama3.2”.

This may take a while and unfortunately {ollamar} doesn’t give you any hint on how long it takes to download all the things. But if you want to be sure that ollama works properly in the background, use this command:

In any case, after the download finishes, you should see something like this:

Specify a model to use in {mall}

Since ollama can serve many models, we should be explicit about which models with which parameters it should use. This can be done via the llm_use function.

Don’t worry too much about the parameters here. Just know that you can tweak them if you ever feel the desire.

Sentiment analysis

Now that all of this stuff is set up, we can run our data against our LLM. Speaking about data, we actually need some. Let’s create a fake tibble then.

Now we can take this data and run it against our LLM to try to determine whether our adjectives are “negative”, “neutral” or “positive”. This is called a sentiment analysis and is available in {mall} via the llm_sentiment() function.

Wow, that was easy. And it was fast too. Since llama3.2 is quite a small model the calculations finished quickly. Of course, larger models may deliver better results but they may also run longer or even require compute resources that your machine doesn’t have. In any case, let’s move on to another use case.

Classification

The next use case is pretty similar. Instead of grouping things into “neutral”, “positive” and “negative”, we can group things into other kinds of categories. Let’s take fruits and assign them to colors.

Well, that wasn’t great. Let’s try to use a larger Llama model like “Llama3.1:8b” which is a bit larger and has 8 billion parameters.

Well, this looks much better. Hoooray!

So what’s the difference between classification and sentiment analysis?

Classification and sentiment analysis are kind of the same. If anything, you can think of sentiment analysis as being a special case of classification. But if you don’t believe me, let’s do a little expedition:

If we take a look into llm_sentiment(), we notice that it’s just a wrapper around llm_vec_sentiment().

And if you follow this trail, you’ll realize that

• llm_vec_sentiment() calls m_vec_prompt()

• m_vec_prompt() calls m_backend_prompt()

• m_backend_prompt() calls m_backend_prompt.mall_session()

and there you’ll find the default prompts for classification and sentiment analysis. You’ll notice that apart from the naming of arguments they are pretty much the same. Here’s the one for classification.

And here’s the one for sentiment.

Looks pretty identical to me. So hopefully this convinces you that classification and sentiment analysis are kinda the same.

Anyway, I wanted to give you a quick glimpse behind the scenes of AI magic. As you can see, many things are nowadays reduced to a simple prompt. For more use cases of how to use {mall}, check out its excellent documentation:

Hope you enjoyed this little intro to the {mall} package. Next week, we’ll be back with more AI stuff. As always, if you have any questions, or just want to reach out, feel free to contact me by replying to this mail or finding me on LinkedIn or on Bluesky.

See you next week,
Albert 👋

Enjoyed this newsletter? Here are other ways I can help you:

• My playlist on YouTube

• Free book: Yet Again - R + Data Science

• Free book: Beautiful tables with {gt}

• Blog posts