Happy March!
This month I am putting a bow on my Q1 roadmap and continuing to unify the two Memphis execution engines.
I chose an update post because I want to “build in public.” What follows is what has been on my mind! I don’t write advice posts (Why you shouldn’t use Vec) or random technical overviews (Vectors vs. Slices in Rust: A Complete Guide) because I am unable to do either with a straight face. If I could, I’d probably still work in a large organization where those kind of ambitious but safe norms are politely celebrated. Someone else will write those pieces and I support them in the same way I support someone running a marathon; I’m not against it, but that doesn’t mean I want to.
It appears this “build in public” includes me breaking the fourth wall on my writing process! Can you tell I struggle in groups? If you’re still here, let’s continue.
Q1 Roadmap
I began Q1 with a list of 6 items I wanted to achieve for From Scratch. As someone who once flew to Chicago to write a list of personal goals on a public library whiteboard with a close friend, I’m no stranger to making goals. What felt new was doing them with that uniquely American fuel: a profit motive.
The Original 6 (the little-known prequel to The Magnificent 7) were:
- Launch website testimonials (thanks Jakub!)
- Improve SEO for fromscratchcode.com
- Launch lead magnet
- Run a small trial with paid ads
- Write chapters 3-5 of my novella
- File taxes and pay Q1 estimated taxes
I would later add two more:
- Add CTA to and modernize my personal site
- Create a runbook for follow-ups
I’m still working on Chapter 5 (Chapters 1-4 are on From Scratch Press) and the follow-up runbook, but everything else is DONE. Who knew you could accomplish things without Jira for Enterprise.
These records have unintentionally produced a fairly detailed account of the playbook I’m running to build my online business. Which I’m excited to share with you in case you too are interested in how to earn small amounts of money with only an internet connection.
I feel pride looking at this list because it makes my efforts feel less random. I can reassure myself Sure, I’m still growing, but here’s my strategy. I can (and have!) thrown this list into ChatGPT and asked what foundational pieces am I missing. And each time it says “Share your work on social media,” I close the tab and search for From Scratch on Google instead.
Multiple Execution Engines
Memphis has supported two execution engines for about a year now. But barely.
The treewalk interpreter is farthest along and, if you wanted to try to run real code, what you should use. I’ve been strengthening the bytecode VM’s foundation and it’s coming along but slowly. Because I have no deadlines, I’m being deliberate to define Memphis capabilities versus treewalk capabilities versus bytecode VM capabilities. Did I mention I have no deadlines?
The unification has proceeded in the following broad strokes.
Common Entrypoint
Initially, you could pick an engine like this.
# Treewalk is default FOR NOW
memphis example.py
# VM selected using an environment variable
MEMPHIS_ENGINE=vm memphis example.py
Which works fine! But after kicking off the runtime, there was no coming back together. Or reunification.
This remains the interface to select a non-default engine, but I’m gradually unifying more of the code behind the scenes. I’ve also floated the idea of using a Rust feature flag instead of an environment variable to produce a smaller binary.
Common Return Type
Because I respect those who came before me, I wanted to treat Python runtime errors as first-class. Meaning I wouldn’t trap them below deck after hitting an iceberg.
Instead of separate error types for treewalk and VM errors, I would combine them. This would also be a chance to turn this dumping ground I aspirationally called InterpreterError into a type which actually represented possible Python runtime errors.
// The treewalk version was first so it is known as "Interpreter" here
pub enum InterpreterError {
Exception(DebugCallStack),
TypeError(Option<String>, DebugCallStack),
KeyError(String, DebugCallStack),
ValueError(String, DebugCallStack),
NameError(String, DebugCallStack),
AttributeError(String, String, DebugCallStack),
FunctionNotFound(String, DebugCallStack),
MethodNotFound(String, DebugCallStack),
ClassNotFound(String, DebugCallStack),
ModuleNotFound(String, DebugCallStack),
DivisionByZero(String, DebugCallStack),
ExpectedVariable(DebugCallStack),
ExpectedString(DebugCallStack),
ExpectedInteger(DebugCallStack),
ExpectedList(DebugCallStack),
ExpectedTuple(DebugCallStack),
ExpectedRange(DebugCallStack),
ExpectedSet(DebugCallStack),
ExpectedDict(DebugCallStack),
ExpectedFloatingPoint(DebugCallStack),
ExpectedBoolean(DebugCallStack),
ExpectedObject(DebugCallStack),
ExpectedClass(DebugCallStack),
ExpectedFunction(DebugCallStack),
ExpectedIterable(DebugCallStack),
ExpectedCoroutine(DebugCallStack),
WrongNumberOfArguments(usize, usize, DebugCallStack),
StopIteration(DebugCallStack),
AssertionError(DebugCallStack),
MissingContextManagerProtocol(DebugCallStack),
RuntimeError,
EncounteredReturn(ExprResult),
EncounteredRaise,
EncounteredAwait,
EncounteredSleep,
EncounteredBreak,
EncounteredContinue,
}
pub enum VmError {
StackUnderflow,
StackOverflow,
NameError(String),
RuntimeError,
}
I landed on this streamlined structure which could now be used on both engines. All my previous ExpectedString, ExpectedInteger, etc., variants are now just a TypeError. This mirrors CPython, where an optional String parameter provides more detail.
pub struct ExecutionError {
pub debug_call_stack: DebugCallStack,
pub execution_error_kind: ExecutionErrorKind,
}
pub enum ExecutionErrorKind {
RuntimeError,
ImportError(String),
TypeError(Option<String>),
KeyError(String),
ValueError(String),
NameError(String),
AttributeError(String, String),
DivisionByZero(String),
StopIteration,
AssertionError,
MissingContextManagerProtocol,
}
With this unified type, I was able to test for an expected NameError in a crosscheck test. Which reminds me, I should really write more about crosscheck, my testing framework for both engines. I have a fun proc macro in the works there.
Now that both engines returned an ExecutionError, I needed to build an interface to push new stack frames to the DebugCallStack. These would be displayed to the user whenever a Python runtime error occurs in their user code.
Common Debug Stack Trace
I’m still actively working on unifying stack traces, but I’m excited because it is me saying “Memphis supports a debug stack trace, regardless of what execution engine you choose,” rather than just “both engines have a stack trace.” Do you see the difference? It’s a pLaTfOrM. Or maybe I mean fRaMeWoRk? It’s something.
I cleaned up my DebugStackTrace and DebugCallStack structs and moved them into a domain module to indicate they represent a Python stack trace, but should NOT be used as a source of runtime info for an engine.
One challenge is giving each engine access to the right-sized shared state object. This would be how each engine could register new stack frames; think something like state.push_stack_frame(function.to_stack_frame()) when entering a new function context. I’m attempting to balance platform capabilities (MemphisState) against freedom of implementation within each engine (TreewalkState and VmState).
The other challenge of implementing stack traces is it forces you to keep around your metadata for use at the right time. Metadata like file paths and line numbers aren’t necessary for actually running Python code, but they’re essential for debugging. When a statement is parsed and immediately evaluated (treewalk) or immediately compiled (bytecode VM, though this has a bug right now), we record the line number of the start of the current function and increment it each time we see a new line. This parser<>runtime communication is only necessary for stack traces (for me. so far.).
This work stream has been a crash course in applying the Single-Responsibility Principle a few decades after I first learned of it. It’s easy to understand the definition (”of course each function/struct/class only does one thing!”). But applying it? That’s harder (”we’ve got access to state here so I’ll put it there!”). The result is my code is GRADUALLY shifting from a medium number of medium-sized functions to a whole lot of tiny ones. I’ve always been an advocate for adding a second entrypoint, either through unit tests or a REPL, but Memphis has forced me to expand that thinking to an entirely different level.
The End
I’m putting the finishing touches on my Q2 roadmap, which should take my business to the max! Do people still say that?
I recently finished The Pathless Path by Paul Millerd and his message of finding the work each of us wants to do indefinitely resonated with me. With my Memphis engine work and my roadmap work, I believe I’m closer than ever to finding that. I’d also love to be a technical mentor to anyone reading this. Because money.
Hope you are well!
Subscribe & Save [on nothing]
Want a software career that actually feels meaningful? I wrote a free 5-day email course on honing your craft, aligning your work with your values, and building for yourself. Or just not hating your job! Get it here.
Build [With Me]
I mentor software engineers to navigate technical challenges and career growth in a supportive, sometimes silly environment. If you’re interested, you can explore my mentorship programs.
Elsewhere [From Scratch]
In addition to mentoring, I also write about neurodivergence and self-employment. Less code and the same number of jokes.
- Rebranding? I hardly knew branding - From Scratch Press