Day 2: The Lexical Analyzer

2026-05-19T00:00:00+00:00

Welcome back to our journey of building WyrdLang from scratch! Last time, we laid the groundwork understanding about what a programming language really is, why we’re building a tree-walk interpreter, and how we’ll follow Crafting Interpreters</em> but with a twist (C++ instead of Java and replaced keywords in the WyrdLang).</p>

Today, we write the first real piece of our interpreter: the lexical analyzer, also called a lexer or scanner. This is where raw source code, strings of characters gets transformed into something the rest of our interpreter can understand.</p>

If the first day was about planning the expedition, today we’re actually starting the climb.</p>

What Exactly Does a Lexer Do?</h2>
When we write code, the computer initially sees nothing but a long, continuous string of characters. It doesn’t inherently know where one idea ends and the next begins. The Lexer’s</strong> job is to scan this string and chop it into Tokens. </p>
The “Token” System: The Lexer acts like a high-speed sorter, labeling every scrap of text it encounters based on its function:</p>

Keywords: It sees if or return and marks them as “Structural Commands.”</li>
Identifiers: It sees user_score or total and marks them as “Names/Labels.”</li>
Operators: It sees + or * and marks them as “Mathematical Actions.”</li>
Literals: It sees 42 or “Hello” and marks them as “Raw Data.”</li>
Punctuation: It sees { or ; and marks them as “Boundaries.”</li> </ul>
Imagine the Lexer encounters this line of code:</p>

x = 10 + y;</p> </blockquote>
It doesn’t just “read” it; it dissects it into a sequence like this:</p>
Character(s)</th> Lexer’s Classification (Token Type)</th></tr></thead>

x</td> Identifier</td></tr>
=</td> Assignment Operator</td></tr>
10</td> Integer Literal</td></tr>
+</td> Addition Operator</td></tr>
y</td> Identifier</td></tr>
;</td> Statement Terminator</td></tr> </tbody></table>
Why This Matters:</p>
If you accidentally type 10abc</strong>, the Lexer is the first line of defense. It looks at its rulebook, realizes that a number followed immediately by letters doesn’t fit any known “Token” pattern, and throws a Lexical Error.By the time the Lexer is done, the messy text has been converted into a clean, organized stream of tokens, ready for the next stage, the Parser: to figure out what those tokens actually mean when put together.</p>
Input</h3>
A string of source code, like this:</p>
enchant health = 100; cast health; </code></pre> Output</h3> A list of tokens – small, labeled chunks that each carry meaning:</p> # High level overview of the token format [ENCHANT] [IDENTIFIER(health)] [EQUAL] [NUMBER(100)] [SEMICOLON] [CAST] [IDENTIFIER(health)] [SEMICOLON] [EOF] </code></pre> Each token remembers:</p> Type</strong> (what kind of thing it is: keyword, number, identifier, operator…)</li> Lexeme</strong> (the exact characters from the source)</li> Literal value</strong> (for numbers or strings, the actual value like 100</code> or "Hello"</code>)</li> Line number</strong> (so we can later point to exactly where an error happened)</li> </ul> The lexer doesn’t care about meaning or grammar, it just recognizes the pieces. It’s like sorting LEGO bricks by color and shape before you start building a something.</p> Output Format in our lexer used in WyrdLang</h3> Type | Lexeme | Literal Value </p> </blockquote> Real Input and Output formats related to our lexer</h4> Input:</li> </ol> enchant health = 100; cast health; </code></pre> Output</li> </ol> // For: enchant health = 100; 32 enchant null // Type 32: Keyword 40 health null // Type 40: Identifier 15 = null // Type 15: Operator 21 100 100 // Type 21: Number (Literal value is the integer 100) 12 ; null // Type 12: Terminator // For: cast health; 33 cast null // Type 33: Keyword 40 health null // Type 40: Identifier 12 ; null // Type 12: Terminator 38 null // Type 38: End of File (EOF) </code></pre> The number reflected in the Type</strong> column actually reflects the enum values of the different token types we have defined in the header.</li> </ul> TokenType.h Header</h3> #pragma once enum TokenType { // Single-character tokens. LEFT_PAREN, RIGHT_PAREN, LEFT_BRACE, RIGHT_BRACE, COMMA, DOT, MINUS, PLUS, SEMICOLON, SLASH, STAR, // One or two character tokens. BANG, BANG_EQUAL, EQUAL, EQUAL_EQUAL, GREATER, GREATER_EQUAL, LESS, LESS_EQUAL, // Literals. IDENTIFIER, STRING, NUMBER, // Keywords. TOGETHER, CLAN, OTHERWISE, NAY, SPELL, CYCLE, WHEN, EMPTINESS, EITHER, CAST, MANIFEST, ELDER, THINE, AYE, ENCHANT, ASLONGAS, END_OF_FILE }; </code></pre> Why Can’t We Just Use Raw Text?</h2> Why do we have to go through all this trouble? Why not feed the source code directly to the parser?</p> Two reasons:</p> 1. Efficiency</h3> The parser doesn’t want to worry about skipping whitespace, handling comments, or checking if =</code> is part of ==</code>. The lexer does all that boring work once, so the parser can focus on structure.</p> 2. Simplicity</h3> Tokenizing turns a messy string into a clean, uniform stream. It’s much easier to write a parser that expects EQUAL</code> tokens than one that has to scan characters like =</code>, !</code>, =</code> again and again.</p> Think of the lexer as your interpreter’s first filter. It removes the noise (whitespace, comments) and labels the signal (keywords, numbers, names).</p> The WyrdLang Tokens: Our Alphabet</h2> Before we can scan, we need to know what we’re looking for. I defined a set of token types that covers everything WyrdLang can say.</p> Single-character tokens</h3> ( ) { } , . ; + - * / </code></pre> These are easy: we see a (</code>, we emit a LEFT_PAREN</code>. We see a ;</code>, we emit a SEMICOLON</code>.</p> One-or-two character operators</h3> ! != = == > >= < <= </code></pre> These need a little lookahead. When we see a =</code>, we check the next character, if it’s another =</code>, we emit EQUAL_EQUAL</code> (the equality operator). If not, it’s just EQUAL</code> (assignment).</p> Literals</h3> IDENTIFIER</code> – names like health</code>, summonDragon</code>, _temp</code></li> STRING</code> – text inside double quotes, like "Hello, world!"</code></li> NUMBER</code> – integers and decimals: 42</code>, 3.14</code></li> </ul> Keywords</h3> Remember our keyword mapping from Day 1</a>? Here’s the complete list as token types:</p> WyrdLang Keyword</th> Token Type</th></tr></thead> enchant</td> ENCHANT</td></tr> spell</td> SPELL</td></tr> clan</td> CLAN</td></tr> when</td> WHEN</td></tr> otherwise</td> OTHERWISE</td></tr> aslongas</td> ASLONGAS</td></tr> cycle</td> CYCLE</td></tr> manifest</td> MANIFEST</td></tr> cast</td> CAST</td></tr> aye</td> AYE</td></tr> nay</td> NAY</td></tr> emptiness</td> EMPTINESS</td></tr> together</td> TOGETHER</td></tr> either</td> EITHER</td></tr> thine</td> THINE</td></tr> elder</td> ELDER</td></tr> </tbody></table> Plus a special EOF</code> token to mark the end of the file.</p> How the Lexer Works</h2> Let me walk you through the scanning of a tiny WyrdLang program:</p> enchant x = 5; </code></pre> I’ll simulate the lexer’s internal state. It keeps:</p> start</code> – where the current token begins</li> current</code> – where we are now</li> line</code> – current line number (starts at 1)</li> </ul> Step 1: Read 'e'</code></h3> We see a letter. That means it could be a keyword or an identifier. We keep consuming characters while they are letters, digits, or underscores. We get "enchant"</code>.</p> Then we check our keyword table, yes, it’s a keyword! Emit ENCHANT</code>.</p> Step 2: Skip whitespace</h3> Space character, ignore, move on.</p> Step 3: Read 'x'</code></h3> Letter again. Consume until we hit a non-identifier character (the space after x</code>).</p> "x"</code> is not a keyword, so emit IDENTIFIER</code> with lexeme "x"</code>.</p> Step 4: Skip whitespace</h3> Space, ignore.</p> Step 5: Read '='</code></h3> Single character =</code>. Peek at next char, it’s a space, not another =</code>. Emit EQUAL</code>.</p> Step 6: Skip whitespace</h3> Space, ignore.</p> Step 7: Read '5'</code></h3> Digit. Consume all digits. No decimal point. Convert "5"</code> to the number 5.0</code>.</p> Emit NUMBER</code> with literal value 5.0</code>.</p> Step 8: Read ';'</code></h3> Single character ;</code>. Emit SEMICOLON</code>.</p> Step 9: End of file</h3> No more characters. Emit EOF</code>.</p> Final token list</h3> [ENCHANT] [IDENTIFIER(x)] [EQUAL] [NUMBER(5.0)] [SEMICOLON] [EOF] </code></pre> That wasn’t so hard, right? Now imagine doing this for thousands of lines of code, that’s what our lexer will do in milliseconds.</p> Handling the Tricky Parts</h2> Not everything is as simple as enchant x = 5;</code>. Real code has comments, strings, and errors. Let’s see how the lexer deals with them.</p> Comments</h3> WyrdLang supports both single-line and multi-line comments.</p> // This is a comment – everything after // until the newline is ignored. </code></pre> /* This is a multi-line comment */ </code></pre> We consume everything until */</code>, even across newlines.</p> The lexer just skips them entirely, they never become tokens.</p> Strings</h3> When we see a "</code>, we enter string mode. We consume characters until the closing "</code>. We also handle escape sequences like \n</code> (newline) and \"</code> (literal quote inside the string).</p> If we reach EOF without finding the closing quote, we report an error, because an unterminated string is a bug in the user’s code.</p> Numbers</h3> We consume digits, then optionally a decimal point and more digits.</p> That’s it, no scientific notation, no hex, just simple numbers. We can always add more later.</p> Unknown characters</h3> What if the source contains a @</code> or #</code>? Those aren’t part of WyrdLang.</p> Our lexer will print an error like:</p> [line 3] Error: Unexpected character. </code></pre> Then it skips the character and keeps scanning. This way, we can find multiple errors in one run, rather than stopping at the first mistake.</p> What the Lexer Does Not Do</h2> It’s important to know where the lexer’s job ends.</p> The lexer does not</strong>:</p> Check that enchant</code> is followed by an identifier (that’s parsing)</li> Make sure parentheses match (parsing)</li> Understand that 5 + "hello"</code> makes no sense (that’s type checking, later)</li> </ul> The lexer is blissfully ignorant.</p> It would happily tokenize:</p> = = = ENCHANT 42 ENCHANT </code></pre> into:</p> [EQUAL] [EQUAL] [EQUAL] [ENCHANT] [NUMBER] [ENCHANT] </code></pre> Even though that’s total nonsense as a program.</p> That’s fine, the parser will catch the garbage later.</p> Writing the Lexer in C++</h2> I’m following the jlox</code> structure from Crafting Interpreters</em>, but I’m writing it in C++.</p> Here’s the skeleton I built:</p> #pragma once #include "TokenType.h" #include "Token.h" #include <string> #include <unordered_map> #include <vector> class Scanner{ private: std::unordered_map<std::string, TokenType> keywords = { {"together", TOGETHER}, {"clan", CLAN}, {"otherwise", OTHERWISE}, {"nay", NAY}, {"cycle", CYCLE}, {"spell", SPELL}, {"when", WHEN}, {"emptiness", EMPTINESS}, {"either", EITHER}, {"cast", CAST}, {"manifest",MANIFEST}, {"elder", ELDER}, {"thine", THINE}, {"aye", AYE}, {"enchant", ENCHANT}, {"aslongas", ASLONGAS} }; std::string source; std::vector<Token> tokens; int start {0}; int current {0}; int line {1}; void scanToken(); void addToken(TokenType _type); void addToken(TokenType _type, std::any _literal); bool isAtEnd(); bool isAlpha(char _character); bool isDigit(char _character); bool isAlphaNumeric(char _character); char peek(); char peekNext(); char advance(); bool match(char _expected); void identifier(); void number(); void string(); public: Scanner(std::string _sourceData); std::vector<Token> scanTokens(); }; </code></pre> The scanTokens()</code> method loops, calling scanToken()</code> repeatedly until we run out of characters.</p> Each call to scanToken()</code> looks at the current character and decides what to do, either a single-character token, or a more complex one like a string or number.</p> The keyword lookup is done with a hash map (unordered_map<string, TokenType></code>). When we read an identifier, we check the map. If found, it’s a keyword; otherwise, it’s a user-defined identifier.</p> Testing the Lexer – Does It Actually Work?</h2> I wrote a few tests to make sure my lexer behaves.</p> Here’s the simplest one.</p> Input</h3> cast "hello world"; </code></pre> </a></p> Another test: operators and numbers</h2> Input</h3> 10 + 20 * 3.5 </code></pre> </a></p> Errors – Because They Will Happen</h2> I deliberately fed the lexer some bad code to see how it reacts.</p> Unterminated string</h3> Input</h4> enchant broken = "Hello, world; </code></pre> The lexer reaches the end of file without finding a closing quote, and continues – it doesn’t crash.</p> </a></p> The token list will be incomplete (no STRING</code> token), but the user gets a clear error.</p> Invalid character</h3> Input</h4> enchant weird = @#$%; </code></pre> </a></p> Then it skips it and keeps going.</p> The rest of the line tokenizes as best it can.</p> Collecting multiple errors in one pass is much friendlier than stopping at the first problem.</p> What’s Next?</h2> With the lexer finished, we have a solid foundation. The rest of the interpreter will consume tokens, not raw characters. That means the parser (coming next) can focus entirely on grammar and structure.</p> In the next post, we’ll build the parser, the part that takes this flat list of tokens and assembles them into an Abstract Syntax Tree (AST). We’ll also start handling syntax errors, missing semicolons, unmatched parentheses, etc…</p> For now, the lexer works, and I’m one step closer to having a real programming language.</p> The Code</h2> I’ve pushed everything to GitHub. you can find it here WyrdLang</a> Feel free to poke around, try breaking it, and open issues if you find something weird.</p> Day 1: Building WyrdLang 2026-05-08T00:00:00+00:00 Welcome to the first post in what I hope will be an interesting journey—building a programming language from scratch. The programming language will be named as WyrdLang</strong>, and over the coming weeks, I’ll be documenting the entire process of implementing a tree-walk interpreter.</p> Why “WyrdLang”? In Old English, wyrd</em> means fate or destiny. And yes, it also sounds like “weird” which is perfectly fitting for a language with unconventional keywords that sounds like words from a fairytale.</p> I won’t be building this language without any reference or a good resource. I’m going to follow the excellent book Crafting Interpreters</em></a> by Robert Nystrom. </p> Crafting Interpreters</h2> This book is a masterpiece, it walks you through building not one, but two</em> complete interpreters for a language called Lox. The first interpreter, jlox</strong>, is a tree-walk interpreter written in Java. It focuses on concepts and correctness, building a clean, understandable implementation. The second interpreter, clox</strong>, is written in C and includes a bytecode compiler and virtual machine for serious performance.</p> I’ll be focusing on the first interpreter the tree-walk approach, but with a twist.</p> The WyrdLang Difference</h2> While I’m following the structure and wisdom of Crafting Interpreters</em>, I’m making some significant changes:</p> 1. C++ Instead of Java</h3> The original jlox is written in Java, but I’m porting it to C++</strong>. Because I’m more comfortable with C++. While Java is an excellent teaching language, I want to work in a language where I can focus on language implementation concepts rather than fighting with unfamiliar syntax. This means I’ll be dealing with manual memory management (or smart pointers), templates instead of generics, and all the quirks that make C++ both powerful and occasionally a maddening disaster.</p> 2. Different Keywords</h3> Here’s one of the unique aspects of WyrdLang. Instead of standard programming keywords, I’m using a different set of keywords inspired by fairy tales. This is mostly for fun and to make the implementation feel more personal, but the core goal remains building a functional, well-designed interpreter.</p> Original Keyword</th> WyrdLang Keyword</th> Meaning</th></tr></thead> var</code></td> enchant</code></td> Declare a variable</td></tr> fun</code></td> spell</code></td> Define a function</td></tr> class</code></td> clan</code></td> Define a class</td></tr> if</code></td> when</code></td> Conditional execution</td></tr> else</code></td> otherwise</code></td> Alternative branch</td></tr> while</code></td> aslongas</code></td> While loop</td></tr> for</code></td> cycle</code></td> For loop</td></tr> return</code></td> manifest</code></td> Return a value</td></tr> print</code></td> cast</code></td> Print output</td></tr> true</code></td> aye</code></td> Boolean true</td></tr> false</code></td> nay</code></td> Boolean false</td></tr> nil</code></td> emptiness</code></td> Null/nil value</td></tr> and</code></td> together</code></td> Logical AND</td></tr> or</code></td> either</code></td> Logical OR</td></tr> this</code></td> thine</code></td> Current object reference</td></tr> super</code></td> elder</code></td> Superclass reference</td></tr> </tbody></table> Here’s what code looks like in WyrdLang:</p> enchant greeting = "Hello, World!" cast greeting spell summonDragon(name) { when (name == "Smaug") { manifest "The dragon awakens!" } otherwise { manifest "A dragon arrives: " + name } } enchant result = summonDragon("Smaug") cast result </code></pre> What is a Programming Language ?</h2> Before we embark on building our own language, let’s step back and understand what a programming language actually is</em> and how it works.</p> At its heart, a programming language is a bridge between human thought and machine execution. When we write code, we’re expressing our intentions about what we want the computer to do, in a form that’s somewhere between natural human language and the raw binary that CPUs understand. Computers only understand sequences of 1s and 0s. That’s it. Everything else—variables, functions, loops, classes is a beautiful illusion we’ve constructed to make programming manageable for human minds.</p> From Text to Execution</h3> Imagine implementing a programming language as climbing a mountain. we start at the bottom with raw source code in our hands, just a string of characters that a pogrammer typed. As we climb up the mountain, we transform this text through various stages, each one revealing more about what the program means</em>:</p> </div> </center> At the peak, we have a complete understanding of what the programmer wants. Then we descend the other side, transforming that high-level understanding into progressively lower-level representations until we reach something the computer can actually execute.</p> The Two Paths: Compilers vs. Interpreters</h3> There are two main approaches to implementing a programming language:</p> Compilers</strong>: </p> Translate our source code into another language—usually machine code—producing an executable file. Think of C, C++, or Rust. We compile once, then run the resulting program many times. It’s like translating a book from English to French: we do the work once, and then French speakers can read it directly.</li> </ul> Interpreters</strong>: </p> Read our source code and execute it directly, without producing a separate executable file. Think of Python, Ruby, or JavaScript. The interpreter translates and executes our code on the fly, line by line (or statement by statement). It’s like having a real-time translator at a conference: they translate as the speaker talks.</li> </ul> In reality, many modern languages blur the line between these categories. Python compiles to bytecode before interpreting it. Java compiles to bytecode, then a JIT (Just-In-Time) compiler translates hot paths to machine code at runtime. It’s complicated, and gloriously so.</p> Building WyrdLang</h2> I’m setting out to build WyrdLang as a tree-walk interpreter</strong> the simpler of the two main approaches to creating an interpreter. Here’s what that means:</p> What is a Tree-Walk Interpreter?</h3> A tree-walk interpreter works by:</p> Scanning</strong> the source code into tokens (lexical analysis)</li> Parsing</strong> those tokens into an Abstract Syntax Tree (AST)</li> Walking</strong> that tree structure, evaluating each node to execute the program</li> </ol> It’s called “tree-walk” because the interpreter literally traverses the tree structure, executing operations as it visits each node. It’s intuitive, relatively simple to implement, and most importantly for learning, we can understand every single piece of it.</p> The downside of the tree-walk approach is that they are slower than more sophisticated approaches like bytecode virtual machines or JIT (Just-In-Time) compilation. But for a learning project and a language inspired by fairy tales, speed isn’t my primary concern.</p> The Journey Ahead</h2> Over the coming weeks, I’ll be documenting my journey building WyrdLang. Each blog post will cover a specific aspect of language implementation:</p> Scanning</strong>: How we break source code into meaningful tokens</li> Parsing</strong>: How we structure those tokens into a syntax tree</li> Evaluating</strong>: How we execute the tree to run programs</li> Variables and Scope</strong>: How we track enchantments and their domains</li> Control Flow</strong>: How we implement when</code> and aslongas</code></li> Functions</strong>: How spells are cast and results manifested</li> Classes</strong>: How clans organize related spells</li> Inheritance</strong>: How clans learn from their elders</li> </ul> Along the way, I’ll encounter bugs, design decisions, and implementation challenges. I’ll share the failures alongside the successes, the confusing moments alongside the revelations.</p> Why Build a Programming Language?</h2> Fair question. Here’s my answer:</p> 1. To Understand How Things Really Work</h3> Building a programming language is the ultimate “peek behind the curtain,” moving you from driving the car to building the engine from scratch. By mastering low-level mechanics like scoping and memory overhead, you stop guessing at performance and start writing better code in every other language.</p> 2. Incredible Learning Experience</h3> Implementing a language tests every programming skill I have:</p> Data structures</strong>: You’ll build hash tables, trees, dynamic arrays</li> Algorithms</strong>: Parsing algorithms, tree traversals, scope resolution</li> Architecture</strong>: How to structure a complex, multi-phase system, usage of design patterns</li> Debugging</strong>: Finding subtle bugs in complex recursive code</li> </ul> By the end of this journey, I guarantee that I’ll be a better programmer.</p> 3. Little Languages Are Everywhere</h3> I will be able to create domain-specific languages: configuration formats, scripting systems for your application, query languages, markup formats. Understanding how languages work makes you better at designing these “little languages” that power so much of modern software.</p> What’s Next?</h2> In the next post, I’ll dive into the first concrete step: scanning</strong>. We’ll learn how to take raw source code just a string of characters and break it into meaningful tokens. We’ll build the lexer that can recognize WyrdLang’s fairy tale keywords and transform text like enchant x = 42</code> into a stream of tokens the parser can understand.</p> We’ll talk about:</p> What tokens are and why we need them</li> How to recognize different types of lexemes</li> Handling whitespace, comments, and errors</li> The theory behind lexical analysis and regular languages</li> </ul> But that’s for next time. For now, I’m setting up my development environment, sketching out the initial architecture, and preparing for the first lines of code.</p> A Note on Open Source</h2> I’ll be sharing all the WyrdLang source code on GitHub as I build it.</p> Inspecting Debug Symbols with the nm Utility 2024-07-18T00:00:00+00:00 When debugging a program, it’s often necessary to understand the layout of the program’s memory and the relationships between different code segments. The nm</code> utility is a powerful tool that allows you to inspect the debug symbols of an executable or object file, giving you valuable insights into the program’s internal workings.</p> What is nm?</h2> nm</code> is a command-line utility that stands for “Name Manager”. It’s part of the GNU Binutils package and is commonly used to display information about the symbols in an object file or executable. The nm</code> utility can be used to inspect the symbol table of a file, which contains information about the names, addresses, and types of variables, functions, and labels defined in the file.</p> Basic Syntax</p> The basic syntax for using nm is as follows:</p> nm [options] filename </code></pre> Where file is the name of the executable or object file you want to inspect. You can specify multiple files by separating them with spaces.</p> Common Options</h3> Here are some common options you can use with nm:</p> -a</code> or --demangle</code> : Demangles C++ symbol names.</li> -C</code> or --check-summaries</code> : Displays summary information for each section.</li> -D</code> or --dynamic-relocs</code> : Displays dynamic relocations.</li> -l</code> or --line-numbers</code> : Displays line numbers for each symbol.</li> -n</code> or --numeric-sort</code> : Sorts symbols numerically by value.</li> -p</code> or --public-only</code> : Displays only public symbols.</li> -r</code> or --reverse-sort</code> : Sorts symbols in reverse order.</li> -S</code> or --symbolic</code> : Displays symbolic information only.</li> -t</code> or --target=<target></code> : Specifies the target format (e.g. elf32, elf64, etc.).</li> </ul> Symbol types</h3> Symbol Type</th> awdadawdawdawd</th></tr></thead> A</td> Absolute Symbol</td></tr> B</td> In the Uninitialized Data Section (BSS)</td></tr> D</td> In the Initialized Data Section</td></tr> T</td> Debugging Symbol</td></tr> N</td> In the Text Section</td></tr> U</td> Symbol Undefined right now</td></tr> </tbody></table> Lower case symbols are Local symbols</li> Upper case sybols is External symbols</li> </ul> For more information see the manual of NM</a></p> </blockquote> Example Usage</h2> Let’s use the nm utility to inspect a simple C program called hello.c. First, we’ll compile the program using gcc:</p> gcc -ggdb hello.c -o hello_debug </code></pre> Now, let’s use nm to display a list of all symbols in the object file:</p> nm ./hello_debug </code></pre> This will produce a list of symbols in the format:</p> </p> Here, we see a mix of global variables, functions, and labels. The T, D, and U prefixes indicate the type of symbol</p> T</code>: Global variable (Initialized)</li> D</code>: Global variable (Uninitialized)</li> U</code>: External reference</li> </ul> nm -n … (Display in Sorted Order)</h4> It will sort using the virtual Addresses</p> </blockquote> nm -n hello_debug </code></pre> </p> nm -g (List External Symbols)</h4> nm -g hello_debug </code></pre> </p> nm -S (display Size)</h4> nm -S hello_debug </code></pre> </p> Demo</h3>

Personal Blog

Day 2: The Lexical Analyzer

Why Can’t We Just Use Raw Text?</h2>
Why do we have to go through all this trouble? Why not feed the source code directly to the parser?</p>
Two reasons:</p>

1. Efficiency</h3>
The parser doesn’t want to worry about skipping whitespace, handling comments, or checking if `=</code> is part of ==</code>. The lexer does all that boring work once, so the parser can focus on structure.</p>`

Handling the Tricky Parts</h2>
Not everything is as simple as `enchant x = 5;</code>. Real code has comments, strings, and errors. Let’s see how the lexer deals with them.</p>`

Unterminated string</h3>

Day 1: Building WyrdLang

Building WyrdLang</h2>
I’m setting out to build WyrdLang as a tree-walk interpreter</strong> the simpler of the two main approaches to creating an interpreter. Here’s what that means:</p>

Why Build a Programming Language?</h2>
Fair question. Here’s my answer:</p>

Inspecting Debug Symbols with the nm Utility

nm -n … (Display in Sorted Order)</h4>

It will sort using the virtual Addresses</p> </blockquote>
`nm -n hello_debug </code></pre> </p> nm -g (List External Symbols)</h4> nm -g hello_debug </code></pre> </p> nm -S (display Size)</h4> nm -S hello_debug </code></pre> </p> Demo</h3>`

nm -g (List External Symbols)</h4>
`nm -g hello_debug </code></pre> </p> nm -S (display Size)</h4> nm -S hello_debug </code></pre> </p> Demo</h3>`

nm -S (display Size)</h4>
`nm -S hello_debug </code></pre> </p> Demo</h3>`

Demo</h3>