Guten Tag! 👋 

Many greetings from Munich, Germany. Well, it’s actually from Cologne, Germany today. But for most of you it will probably be same same anyway.

Today, let us master Mirai magic. I call it 3M. Impressive what fancy naming my PhD-level pre-trained neural network I call brain can come up with, right? 🤪

In case you didn’t know, the {mirai} package is a fabulous package to make R run asynchronously. This is particularly helpful when you want to run things in parallel that are not really resource-intensive.

For example, if you want to make many API calls (like chatting with an LLM), using {mirai} can be a good idea. After all, with API calls your R session isn’t doing anything. It’s just waiting for the response. These type of I/O-operations are best run async. So let’s find out how to do that with Mirai.

A simple starter

It’s pretty easy to get started with Mirai. All you have to do is to wrap your slow operation into mirai(). We’re just going to simulate an API call with wait time.

Nice! Did you notice it? We were supposed to wait 4s but the mirai() call finished immediately.

And sure…we don’t see any result of our calculation. But who needs that, right? As long as it’s fast, we’re all good, right!? (Reminds me of this terrible joke, lol)

Getting results

Alright, fine. You got me there. Your boss-person might actually care about results. Maybe not so much that it’s the only thing that matters and you’re allowed to work from anywhere as long as you deliver. But results are important at least a bit.

So let’s deliver some results. The way to do that is to assign the mirai result to a variable and then fetch the result using Mirai’s specical [] syntax.

So how is this faster?

As you can see, we had to wait 4 seconds in the end (even looks like 5 due to rounding). But the calculation actually ran in the background. It’s just hard to see here because we’re not doing anything with our freedom. But watch what happens if we waste our time for a tiny bit of doom-scrolling waiting again.

As you can see our code now had 7s of waiting in total but still everything finished after 4 seconds. That happened because the initial 4s were happening in the background.

So R was concurrently waiting 3s in the main session and 4s in the background. That still leaves us at 4s in total because waiting happened at the same time and not sequentially. Pretty sweet, isn’t it?

Many calculations

So how about running many and not just two calculations at once? Well, Mirai let’s you iterate using the mirai_map() function:

Oh no! This did not work because we have not set up concurrent runners for the iterations. We have to specify how many runners can run at the same time. As the error message implies, we do that via the daemons() function:

So now this works. And once again, we assigned this to a variable and can collect the result with brackets.

See how all of our iterations finished at the same time? Each iteration was to wait 3s. But they did that in parallel thanks to Mirai.

The output of the iteration is a list by default. But there are special attributes we can use in these brackets to flatten that into a vector if possible.

Closing the daemons

After you’ve fetched the results, you should also get rid of the background processes. Just set the number of daemons to zero.

Nice! You now know about the basics of Mirai. Next week, let’s do some trickier things with it. This will include a few things that will mess you up if you never heard about that before.

Hope you enjoyed this week’s newsletter. As always, feel free to hit the reply button to share your thoughts.

Happy Coding and see you next week,
Albert

Whenever you’re ready, there are three I can help you:

1. Automate Your Data Reports: This course helps data analysts eliminate manual copy-paste reporting by automating PDF reports end-to-end, saving hours every cycle and preventing costly mistakes. (Using the lovely Typst language 😍)

2. Generate Insights in Minutes, not Hours: This comprehensive course teaches you to handle data faster, smarter, and more efficiently.

3. Bespoke Data Science Solutions: I’ve helped clients build their own data science solutions. Whether building custom web apps, PDF reports, AI automations or teaching workshops, I’ve got you covered. You can reach out to me via this form (or simply hit reply to this email)