I’m developing some ideas around using ensembles of LLMs for specific tasks. Today I’m sharing a preview of the first “trivial” example, which is the foundation of linking arrays of LLMs to the concept of ensembles from statistical mechanics.
In this example, an ensemble of LLMs are called with the same prompt, but “high” temperature (of 0.99). There is a “prompt” and a “grading criteria” that are used to create the full prompt.
LLM Ensembles - Trivial Example
The only variance is the from the “temperature”, which you can see doesn’t result in much variety.
Then the “judge” is asked to rank the results, based on the grading criteria, with the judge having a low temperature (0.1):
The output is not impressive. But from here we can start to build up to more interesting examples.
Basically, it adds a bunch of functions to Sheets that help interface with OpenAI (in both directions). Every cell can run it’s own API call (in parallel). You can reference other cells. Combine, list, split, it’s really nuts.
If LLMs weren’t already capable of generating more text than anyone could possibly read or use, this really seals the deal. Quantity: Check ✅. See below for about 20 question and answer pairs generated using this plugin in under 1 minute, from my initial input of just 4 questions (and no answers).
Q&A Pairs, as far as the eye can see.
Quality: In Progress ⌛. This is more work. Especially if there is too much content to have a human editor. Stay tuned. 📻
Here’s a “deep work” 1,2 punch that I have had some good results with.
Walk and talk out a very clear plan. Record a voice memo. I use the Yealink BH71 Pro headset, which has amazing noise cancelling (for the microphone) and I look like a telemarketer on a hike. Later you can transcribe this with your favorite voice transcription tool. (Optionally, get ChatGPT to format the transcription for you.) It is very important to resist the urge to use your phone, just talk out the steps, talk out the framework, talk out the plan for what you will do. When you get distracted, bring it back to the plan. Make sure you have a clear outcome in mind.
Block 90-120 minutes of “deep work” time to execute the plan. Try to get to the outcome, try to follow the steps. Maybe the plan was too ambitious, maybe you need to adjust the plan. But try to get to the outcome. Maybe you don’t need 99% of the structure you thought that you did (that’s what happened to me today). But you incept in yourself the idea of what you want to do, and then you do it. When you get distracted, bring it back to the outcome.
That’s it. Go get ’em, champ!
Go get 'em, champ.
P.S. - I came up with an interesting theory while performing Step 1 this morning (a distracting thought from the plan I was formulating), which is that Cal Newport is actually an AI being sent to us from the future to teach us how to focus like a machine. (Think: The Terminator of “Deep Work”.) I’m not sure how to test this hypothesis.
After watching it, I realized that many of the AI applications where I thought I would need plug-ins, or special function calling set-ups, or a proprietary/paid solution, etc — these can be solved with better prompting. It’s also helpful to see them build up to more complex use cases, step-by-step.
One of the points that Andrew Ng makes in the intro is that there is no “best prompt for X”. So instead this course teaches you the fundamentals, and more importantly - how to iterate to get to a solution that works for your application.
With these prompting fundamentals and a basic RAG pipeline (which is getting easier and easier every day) - you can really accelerate a ton of business tasks.
I am now convinced that there is something for everybody.
Clearly someone likes this, there is only one left in the box.
I could imagine a point in time where I would be excited to identify as a “slime-licker”. But licking toxic waste? No thank you.
P.S. - In case this is for you, a quick search seems to indicate that this particular form factor of “sour rolling liquid candy” is no longer for sale online. (Hit me up and I’ll tell you where the physical location of this last roller is.) But don’t worry - you can still squeeze various flavors of toxic waste onto your tongue with two-day delivery 😛.
Here are some lessons learned from today’s attempts into social media automation workflows:
When soliciting creative input from Language Learning Models (LLMs), consider asking for more than four variations. This is particularly useful if the workflow involves human selection at some point. (It makes sense that most of the generative image tools do it this way.)
I’ve decided to move away from IFTTT as it no longer serves my needs. I’ve transitioned to using Postman for simple webhooks and Make.com for more intricate routing.
Currently, I’m utilizing Buffer. However, they’ve ceased the addition of new apps to their API, forcing me to use Make > Buffer. In the future, I might consider posting directly to social media via the individual platform APIs. This is especially feasible with function calling and having LLMs write the function calls for me. I’m finding that API-replacement apps like Zapier, Make.com, and IFTTT are becoming more of a hassle than they’re worth.
A significant time-saving tip: use GPT-4 to generate regex patterns for you.
Today was my kickoff call with Athena. I’ve been really impressed by the intentionally of their onboarding process.
Each step of their sales funnel was very clear and effective. Each stage of the on-boarding has a clear purpose and structure. The first 100 days are planned out in detail, which provides a nice framework and sense of certainty and security.
I’m excited to see how it goes! I will keep you posted.
Are you at the cutting edge of your field? Or are you at the bleeding edge?
Most of the time, I think I’m at the cutting edge. But then I notice that I’m bleeding.
I get so caught up in the excitement and novelty that I don’t realize that I have crossed over to the bleeding edge. It’s hard to get paid at the bleeding edge, even harder to be profitable. It’s even more exciting than the cutting edge, and even more draining.
In silicon valley, the saying is “pioneers get arrows in their backs, settlers get land.” This is the distinction that I’m trying to make between the cutting edge and the bleeding edge. Both the pioneers and the settlers had the same vision, the same excitement. One was just a little too soon (and it’s really hard to know when the timing is right).
Part of what originally drew me to 3D printing was the theme of commuting between the digital and physical realms. When I first started, I knew nothing about CAD, a category of software that continues to frustrated me to this day. (I’m just not into graphical programming languages.)
There are a few cool new tools that I’ve been playing with that make commuting between the physical to digital worlds a little easier. Two that I’m really into right now are Commonsense Machines (CSM.ai) and Luma AI from LumaLabs. Both companies offer text-to-3D and video-to-3D. CSM also offers 2D-to-3D, which is getting better every month.
A couple of Christmases ago, I accidentally ended up on the bleeding edge of web-based Augmented Reality. What I thought would be pretty straightforward involved hiring and firing at least 3 different professional WebAR developers. But we ended up building polySpectra AR - which I still think is pretty nifty. The idea was to give users a free massless preview of their .STL, before they would upload it to a 3D printer or 3D printing service.
Fast forward to this morning, something clicked and I realized that we could pretty quickly tweak polySpectra AR to give users a completely CAD-free workflow to both visualize 3D models and then manufacture them with 3D printing.
I’ve spent a lot of time banging my head against the wall trying to quickly get started with a bunch of new AI tools recently. The quickstart guides have not been so quick.
In particular, it was interesting to see the Poe team watch their quickstart get (mis)interpreted in real time at the Hackathon on Saturday. There were a few really common hiccups where people got stuck, even though it was clearly in the docs, almost every single team got stuck at the same problem and had to ask for help.
I definitely have a new appreciation for the importance of a good quickstart guide. Today I decided that we should build one for polySpectra. We started with our go-to printer: the Asiga Pro 4K.
By using a template with common variables, we can now pretty quickly generate a new quickstart guide for any printer we support. The first one took about 3 hours, the second one about 10 minutes.
I really think one of the most important superpowers of AI tools is making it easier for people to interact with code.
I’m also getting into Zoom Clips, which means I don’t need to pay Loom and Zoom.
