An underqualified reading list about the transformer architecture

2025-10-30T00:00:00+00:00

When I see a list of learning resources, I assume that whoever curated it knows much more about the topic than what I’ll learn. So you wouldn’t be blamed for thinking that I have read 100x the amount of things about the transformer architecture than what I’m sharing here. In reality, that multiplier is shamefully close to 1x. Put another way: I have no idea what I’m talking about.

OPCODE</td>	Before Berlin</td>	After Berlin</td> </tr>
SLOAD</td>	800</td>	2100</td>	100</td> </tr>
SSTORE</span> from 0 to 1</td>	20000</td>	22100</td>	20000</td> </tr>
SSTORE</span> from 1 to 2</td>	5000</td>	5000</td>	2900</td> </tr>
SLOAD + SSTORE*</span></td>	5800</td>	5000</td>	3000</td> </tr>
SSTORE* + SLOAD</span></td>	5800</td>	5100</td>	3000</td> </tr>
SSTORE</span> of an already written slot</td>	800</td>	100</td> </tr>
From a non-zero value to a different non-zero value, like in the third row </span> </td> </tr> </tbody> </table> </div> Notice that in the last row it doesn’t make sense to talk about the slot having been accessed or not because, if it was previously written, then it was also accessed.</p> EIP-2930: Optional Access List transactions</h2> The other EIP we mentioned at the beginning was EIP-2930. This EIP adds a new type of transaction that can include an access list in the transaction payload. This means that you can declare beforehand which addresses and slots should be considered as accessed before the transaction’s execution starts. For example, an `SLOAD</code> of a non-accessed slot costs 2100, but if that slot was included in the transaction’s access list, then that same opcode will cost 100.</p>` `But if the gas costs are lower when an address or storage key is already accessed, does this mean that we can add everything to the transaction’s access list and get a gas reduction? Yay, free gas! Well, not exactly, because you also need to pay gas for each address and each storage key that you add.</p>` `Let’s see an example. Say we are sending a transaction to contract A</code>. An access list could look like this:</p>` accessList: [{ </span> address: </span>"<address of A>"</span>, </span> storageKeys: [ </span> </span>"0x0000000000000000000000000000000000000000000000000000000000000000" </span> ] </span>}] </span></code></pre> If we send a transaction with this access list, and the first opcode that uses the 0x0</code> slot is a SLOAD</code>, it will cost 100 instead of 2100. That’s a gas reduction of 2000. But each storage key included in the transaction’s access list has a cost of 1900. So we only save 100 of gas. (If the first opcode to access that slot is an SSTORE</code> instead, we would save 2100 of gas, which means that we’d save 200 of gas in total if we consider the cost of the storage key.)</p> Does this mean that we always save gas when using transaction’s with access lists? Well, no, because we also pay gas for the address in the access list ("<address of A>"</code> in our example).</p> Accessed addresses</h2> So far, we’ve been talking only about the SLOAD</code> and SSTORE</code> opcodes, but those aren’t the only ones that change after Berlin. For example, the CALL</code> opcode had a fixed cost of 700. But after EIP-2929 its cost is 2600 if the address is not in the access list and 100 if it is. And, like the accessed storage keys, it doesn’t matter what OPCODE accessed that address before (for example, if an EXTCODESIZE</code> was called first, then that opcode will cost 2600, and any subsequent EXTCODESIZE</code>, CALL</code>, STATICCALL</code> that uses the same address</strong> will cost 100).</p> How is this affected by transactions with access lists? For example, if we send a transaction to contract A</code>, and that contract calls another contract B</code>, then we can include an access list like this:</p> accessList: [{ address: </span>"<address of B>"</span>, storageKeys: [] }] </span></code></pre> We’ll have to pay a cost of 2400 to include this access list in the transaction, but then the first opcode that uses the address of B</code> will cost 100 instead of 2600. So we saved 100 of gas by doing this. And if B</code> uses its storage somehow and we know which keys it will use, then we can also include them in the access list and save 100/200 of gas for each one (depending on whether the first opcode is an SLOAD</code> or an SSTORE</code>).</p> But why are we talking about another contract? What happens with the contract that we are calling? Why don’t we do this?</p> accessList: [ </span> {address: </span>"<address of A>"</span>, storageKeys: []}, </span> {address: </span>"<address of B>"</span>, storageKeys: []}, </span>] </span></code></pre> We could do it, but it wouldn’t be worth it because EIP-2929 specifies that the address of the contract that is being called (that is, tx.to</code>) is always included in the accessed_addresses</code> list. So we are paying 2400 more for nothing.</p> Let’s analyze our example of the previous section again:</p> accessList: [{ </span> address: </span>"<address of A>"</span>, </span> storageKeys: [ </span> </span>"0x0000000000000000000000000000000000000000000000000000000000000000" </span> ] </span>}] </span></code></pre> This will actually be wasteful unless we include several storage keys more. If we assume that a storage key is always used first by an SLOAD</code>, then we need at least 24 storage keys just to break even.</p> As you can imagine, analyzing this and creating an access list by hand is not fun. Luckily, there is a better way.</p> The eth_createAccessList</code> RPC method</h2> Geth (starting from version 1.10.2) includes a new eth_createAccessList</code> RPC method that you can use to generate access lists. It is used like eth_estimateGas</code>, but instead of a gas estimation, it returns something like this:</p> { </span> </span>"accessList"</span>: [ </span> { </span> </span>"address"</span>: </span>"0xb0ee076d7779a6ce152283f009f4c32b5f88756c"</span>, </span> </span>"storageKeys"</span>: [ </span> </span>"0x0000000000000000000000000000000000000000000000000000000000000000"</span>, </span> </span>"0x0000000000000000000000000000000000000000000000000000000000000001" </span> ] </span> } </span> ], </span> </span>"gasUsed"</span>: </span>"0x8496" </span>} </span></code></pre> That is, it gives you the list of addresses and storage keys that will be used by that transaction, plus the gas consumed if the access list is included</em>. (And, like eth_estimateGas</code>, this is an estimation; the list could change when the transaction is actually mined.) But, again, this doesn’t mean that this gas will be lower than the gas used if you just send the same transaction without an access list!</p> I suppose we’ll discover over time what’s the proper way of doing this, but my pseudo-code guess is this:</p> let </span>gasEstimation </span>= </span>estimateGas</span>(</span>tx</span>) </span>let </span>{ </span>accessList</span>, </span>gasUsed </span>} </span>= </span>createAccessList</span>(</span>tx</span>) </span>if </span>(</span>gasUsed </span>> </span>gasEstimation</span>) { </span> </span>delete </span>accessList</span>[</span>tx</span>.</span>to</span>] </span>} </span>tx</span>.</span>accessList </span>= </span>accessList</span>; </span>sendTransaction</span>(</span>tx</span>) </span></code></pre> Unbricking contracts</h2> It is important to mention that the main purpose of Access Lists is not to use gas. As the EIP explains it</a>:</p> Mitigates contract breakage risks introduced by EIP-2929, as transactions could pre-specify and pre-pay for the accounts and storage slots that the transaction plans to access; as a result, in the actual execution, the SLOAD and EXT opcodes would only cost 100 gas: low enough that it would not only prevent breakage due to that EIP but also “unstuck” any contracts that became stuck due to EIP 1884.</p> </blockquote> This means that if a contract makes assumptions about the cost of executing something, the increase in gas costs could break it. For example, a contract makes a call to another contract like someOtherContract.someFunction{gas: 34500}()</code> because it assumes that someFunction</code> will use exactly 34500 of gas, then it will break. But if you include the proper access list in the transaction, then the contract will work again.</p> Check it yourself</h2> If you want to test this yourself, clone this repo</a> which has several examples that can be executed using Hardhat</a> and geth. Check the README for the instructions.</p> Was this fun?</h2> If this low-level mumbo-jumbo is your idea of fun, you should know that Nomic Labs</a> is hiring</a>.</p> References</h2> EIP-2929</a> and EIP-2930</a> are the two Berlin EIPs relevant to this article.</li> EIP-2930 depends on another part of Berlin: EIP-2718</a>, also called Typed Transactions.</li> EIP-2929 refers to EIP-2200</a> a lot, so if you want to understand gas costs more in depth, you should start there.</li> For more complex examples comparing how gas usage changes, check this</a>.</li> </ul>

But I think this list can be useful anyway, because it has a couple of somewhat idiosyncratic goals: conceptual understanding and historical perspective. By conceptual understanding I just mean having a slightly better mental model of what’s going on under the hood when you use a tool like ChatGPT. And by historical perspective I mean understanding how this architecture fits in the bigger narrative of artificial intelligence research.

Having those goals means that this list is not practical: there’s nothing about prompt engineering or how to use an API, let alone how to implement or train an LLM, or even run one locally. There are also no philosophical musings like “well, but is this really thinking?” nor, God forbid, any ethical discussions about alignment or whatever. But if you want to get a basic understanding of how transformers work and have a (short, incomplete, probably wrong) narrative of how they came about, then I’d say this post is for you.

You and Your Shell

2023-09-24T00:00:00+00:00

This article is my attempt to explain what is a shell and what it does. My goal is for you to build a useful mental model of what’s going on under the hood when you open a terminal, paste some command, and hope that it doesn’t erase all your terrible, embarrasing, hidden sci-fi poems.

Is this for you?</h1>
I think you won’t get much from this article if you can answer these questions without even blinking:

What is the difference between `echo foo</code> and echo 'foo'</code>?</li>`
`What’s going on when you do DEBUG='' some-command</code></li>`
`What’s the difference between a program and a shell built-in? Why do we have both?</li> </ul>`But you may still want to jump directly to the examples</a>. If you find any of them surprising, then maybe there’s something useful for you here! Contents</h1> What is a shell?</a> Running programs</a></li> The worst shell in the world</a></li> Finding programs</a></li> Parsing arguments</a></li> Globs</a></li> Quotes</a></li> Environment variables</a></li> Built-in commands</a></li> </ul> </li> Examples!</a> Example 1: The --</code> separator</a></li> Example 2: Escaping quotes</a></li> Example 3: Conditions in bash programming</a></li> </ul> </li> Clarifications</a></li> Further reading</a></li> </ul> What is a shell?</h1> My favorite definition comes from the classic The UNIX Programming Environment</a>: [The shell is] the program that interprets your requests to run programs. </blockquote> This short sentence tells us a lot: First, a shell is just a program. bash</code>, zsh</code> and fish</code> are all different programs that function as shells.</li> Second, a shell interprets requests. You use a specific language for these requests, even if you are not aware of it.</li> Finally, a shell is used to run programs. You can do other things with it, but that’s its main purpose.</li> </ul> Let’s explore this definition in more detail. Running programs</h2> The first thing we need to understand is that a program is always started from another program.1</a> Who started that other program, you might ask? Well, of course: a third program. And who started that one? Yet another program. But who...? I don’t know, a turtle or something</a>. The details of what exactly a program does to start another one are not relevant for our purposes. We can just assume that there is a function that looks like this: function runProgram( pathToProgram: string, arguments: string[], environmentVariables?: Record<string, string> ) </code></pre> This is a crude simplification, but it will do. At its core, starting a program requires its path in the filesystem and a list of arguments. We can also optionally pass a dictionary of environment variables. With this function and our previous definition, we can try to write a very simple and useless shell. The worst shell in the world</h2> Since we defined a shell as a program that interprets requests to run programs and assumed we have a runProgram</code> function, we can write an extremely simple shell that just runs whatever program is passed to it: while (true) { const pathToProgram = readLine(); runProgram(pathToProgram, []); } </code></pre> This would be extremely useless, but it does the job. You input a path to the program you want to run, our shitty shell interprets your request verbatim, and that’s it. But a real shell does much more than that. Finding programs</h2> When you want to run the df</code> program to see if you have enough space to keep downloading</del> buying movies, you don’t write /usr/bin/df</code>, you just write df</code>. This wouldn’t work on our dumb implementation, but it does in any real shell. The reason this works is that the shell will check a list of pre-defined directories to see if any of them contains a program with that name. You probably already know that this list is in the PATH</code> environment variable</a>. We can easily add this feature to our previous snippet, especially if we assume that we have more magic functions available: while (true) { const program = readLine(); const pathToProgram = findProgram(process.env.PATH, program); if (pathToProgram) { runProgram(pathToProgram, []); } else { console.error(`No program named '${program}' was found`); } } </code></pre> Now we are doing some interpretation. But what if we want to pass some arguments to a program? If we run rm Titanic.mkv</code> we’ll get this: No program named 'rm Titanic.mkv' was found </code></pre> Which makes sense! We are taking the full line and interpreting it as a program name. Whitespace is not special, and neither are you. What we need here is some more of that sweet interpretation. Parsing arguments</h2> Let’s update our shell to interpret whitespace as a separator between the name of the program and the arguments: while (true) { const line = readLine(); const [program, ...args] = line.split(/\s+/g); const pathToProgram = findProgram(process.env.PATH, program); if (pathToProgram) { runProgram(pathToProgram, args); } else { console.error(`No program named '${program}' was found`); } } </code></pre> If this code actually worked and we ran rm Titanic.mkv</code> on it, it would actually delete the file. Probably. In any case, the takeaway is that we are doing a more complex interpretation of the command we receive. The command is still just a string, but we are starting to parse it in a somewhat interesting way. Globs</h2> What if you want to remove all the files that end in .mkv</code> in a directory? It’s easy with our shell: you just run ls</code>, then write rm </code>, and then manually add all the files that end with that extension. I’m sure you won’t make any typos. If you don’t like typing, then there are some ways this could be automated. Maybe you could use a version of rm</code> that, instead of a list of files, receives an extension as its argument and removes all the files with that extension. Or perhaps you could have one program that writes all the files with some extension to a file, and then pass that file to another program which consumes it and removes all the files pointed by it... ...or you could just add a new feature to the shell. You could say that, if the shell finds something like .mkv</code>, then it will interpret it as “all the files in the current directory that end in mkv</code>". This is called glob expansion or globbing</a>. Globs</a> are like regular expressions, but worse, or better; it’s hard to say. The important part here, and probably the most important part of this article, is that this expansion is performed by the shell! This is quite easy to check by writing a program that just prints its arguments. With Node.js, you can do something like this: // index.js const arguments = process.argv.slice(2); console.log(JSON.stringify(arguments)); </code></pre> If you run node index.js 1 2 foo</code>, you will get the list ["1","2","foo"]</code>. (Don’t pay too much attention to the .slice(2)</code> part, this just avoids printing the paths to the node binary and the script, which are included in the arguments list for what I’m sure are very good reasons.) If you now create some files: $ touch a.txt b.txt c.txt </code></pre> And then run node index.js .txt</code>, the program will print ["a.txt","b.txt","c.txt"]</code>, not [".txt"]</code>. Things like </code> are called metacharacters. These are characters that receive a special treatment by the shell. There are a lot of these, like ?</code>, |</code>, <</code> and ></code>. But now you, being a smart and curious person, are wondering: “what if I actually want to pass .txt</code> as an argument to a program, for my own devious purposes?". Quotes</h2> This is one of the things you might do without fully understanding. You run something in the shell; it doesn’t work and you get a scary error message. You try adding some quotes. Now it works. Success! But what did actually happen? Here I’m going to quote from The UNIX Programming Environment again. Given the number of shell metacharacters, there has to be some way to say to the shell, “Leave it alone”. The easiest and best way to protect special characters from being interpreted is to enclose them in single quote characters: $ echo '' $ </code></pre> </blockquote> Honestly, you should just close this tab and go read Chapter 3 from that book. It’s so good. But yes: you use single or double quotes to escape metacharacters. Single quotes escape everything, double quotes allows certain expansions within them. I think a good rule of thumb is “always use single quotes unless you know what you’re doing”. So, going back to the question in the previous section, if you want to literally pass .txt</code> as an argument to a program, you can do: $ rm '.txt' </code></pre> And if you have a file that is literally named .txt</code> then 1) you are a monster and 2) it will be removed. You can also try this with the little script we wrote before: $ node index.js '.txt' ["*.txt"] </code></pre> Environment variables</h2> So far, we’ve called our runProgram</code> function without passing any environment variables. The default behavior in this case is that the environment of the parent process will be passed to the child process (or maybe some of them are changed? I’m fuzzy on the details here). What if you want to run a program with some specific environment variables? Most shells let you do this with a special syntax. In bash/zsh/fish, for example, you can do: $ FOO=1 BAR=2 program arg1 arg2 </code></pre> That is, you can prepend the program name with a list of environment variables. As before, it’s easy to write a little script to test this out: // index.js console.log(process.env.FOO); </code></pre> Running node index.js</code> will print undefined</code> but running FOO=1 node index.js</code> will print 1</code>. There is an alternative way to do this that you’ve surely seen before, but which is not exactly equivalent: $ export FOO=1 $ node index.js 1 </code></pre> The difference here is that export FOO=1</code> is modifying the environment of the shell, which is then passed down to our script. This is persistent: if we run node index.js</code> a second time in the same shell session, it will print 1 again. But if we do that with the previous version (assuming a fresh shell instance), it won’t: $ FOO=1 node index.js 1 $ node index.js undefined </code></pre> Built-in commands</h2> The most interesting part of the previous section is export FOO=1</code>. This is an example of running a command in the shell that doesn’t execute a program. export</code> here is what’s called a built-in command of the shell. When you run things like df</code>, rm</code>, vim</code> or whatever, there is an actual file in your disk with the code for that program. You can use whereis</code> to find the path to that file: $ whereis df df: /usr/bin/df /usr/share/man/man1/df.1.gz </code></pre> But if you do the same with a built-in, you get a sad void: $ whereis export export: </code></pre> Why do we have built-ins? Why isn’t everything a program? In many cases, the reason is that the built-in does something that can’t be done by a program. For example, export</code> modifies the environment of the shell and cd</code> changes its current directory. This is something that can’t be done by a program, because a program can’t make these modifications to its parent process.^{2</a>} In any case, if you want to know if a command is a built-in or not, you can use the type</code> command: $ type export export is a shell builtin </code></pre> Type is, of course, a builtin: $ type type type is a shell builtin </code></pre> Examples!</h1> Ok, now for the fun part. Let’s use what we learned to explore some interesting examples. Example 1: The --</code> separator</h2> Let’s start with an easy one. Sometimes you run a command that, in turn, runs another command. For example, let’s say you are working on a Rust command line program. You can run it during development with cargo run</code> and pass arguments to it. cargo run someArg </code></pre> In this case, your program will receive someArg</code> as an argument. Now imagine you added support for a new --foo</code> flag and you want to try it out. If you do this: $ cargo run --foo someArg </code></pre> you’ll get an error: error: Found argument '--foo' which wasn't expected, or isn't valid in this context If you tried to supply '--foo' as a value rather than a flag, use '-- --foo' </code></pre> What’s happening here is that cargo thinks that --foo</code> is a flag for cargo run</code>, not for your program. A similar thing happens when you do npm run some-script</code> and want to pass some arguments to the command executed by the script. In both cases, the solution is to add a --</code>. If we run cargo run -- --foo</code>, then it will work fine. --</code> here is a special argument that changes how Cargo (or npm) interprets the rest of its arguments. What does the shell do with --</code>? Hopefully the answer is obvious: nothing. This is not a metacharacter. Using --</code> is just a convention used by many programs to change how they interpret their arguments; the shell doesn’t care about it. Example 2: Escaping quotes</h2> If you want to pass an argument that actually contains a quote, you can escape it with the other type of quote. That is: $ echo 'The "quick" brown fox' The "quick" brown fox $ echo "The 'quick' brown fox" The 'quick' brown fox </code></pre> If you want to include both kinds of quotes, you could do this: $ echo "The 'quick' brown \"fox\"" </code></pre> But, as mentioned before, it’s better to use single quotes by default. The problem is that you can’t escape single quotes with a backslash in bash and zsh (you can in fish, but let’s ignore that). So, if you want to pass a parameter that contains both double quotes and single quotes and you want to use single quotes for most of it, you have to do this: $ echo 'The '"'"'quick'"'"' brown "fox"' The 'quick' brown "fox" </code></pre> The key to understand this monstrosity is that you can concatenate quotes strings. All of these are equivalent: $ echo foobarbaz foobarbaz $ echo "foo""bar""baz" foobarbaz $ echo 'foo'"bar"'baz' foobarbaz </code></pre> If that makes sense, and if you squint hard enough, then you can understand the previous example. Here it is again, which each part in a different line: $ echo 'The '\ > "'"\ > 'quick'\ > "'"\ > ' brown "fox"' The 'quick' brown "fox" </code></pre> For a good, alternative explanation of this, check this answer</a> on StackOverflow. Example 3: Conditions in bash programming</h2> So far everything we’ve seen is related to running programs. But the language interpreted by shells like bash is pretty much a full programming language, including things like conditionals. For example, you can do: if [ 3 -gt 1 ] then echo "3 is greater than 1" else echo "Math is broken" fi </code></pre> Now, you might think that the syntax for an if</code> statement is something like this: if [ <conditions> ] then <true> else <false> fi </code></pre> Where <conditions></code> uses a terrible syntax. But the actual syntax is this: if <command> then <true> else <false> fi </code></pre> Where <true></code> will be executed if the command exits with a zero status code, and <false></code> will be executd if the command exits with a non-zero code. Wait, does that mean that...? $ whereis [ [: /usr/bin/[ /usr/share/man/man1/[.1.gz </code></pre> Yes: [</code> is a program.3</a> It receives a list of arguments that are interpreted as conditions, and exits with a zero exit code if its arguments evaluate to true, and non-zero otherwise. Actually, [</code> is kind of an alias to the test</code> command. $ test 3 -gt 1 $ echo $? 0 $ test 3 -gt 3 $ echo $? 1 </code></pre> (Here we are using the $?</code> variable, which contains the exit code of the last command, to check the result of the test</code> command.) And I said “kind of an alias” because [</code> is not exactly the same as test</code>. $ [ 3 -gt 1 bash: [: missing `]' </code></pre> As you can check by running man test</code>, [</code> is pretty much the same as test</code>, except that it requires a ]</code> as its last argument. Fun stuff. Clarifications</h1> There are a lot of simplifications in this article. Some of them are there to make the explanation easier to understand, and some of them are due to my own ignorance. This is a list of the ones I’m aware of and that I think are important: Everything here is very UNIX specific. I think the core ideas apply to non-UNIX operating systems, but many of the details are probably different.</li> I didn’t mention system calls at all, even if they are relevant for a more common definition of a shell: “a shell is a program that provides an interface to the operating system”. This is probably a technically better definition, but I wanted to focus on the “interprets requests to run programs” part.</li> The environment is actually a list of strings, not a dictionary. These strings, by convention, look like key=value</code> and are interpreted as a dictionary by programs and standard libraries.</li> </ul> Further reading</h1> The UNIX Programming Environment</a> is a classic. It’s a bit dated, but it’s a great read anyway. Advanced Programming in the UNIX Environment</a> is a more modern and technical book. I haven’t read it cover to cover, not even 10% of it, but it’s great as a reference. Footnotes</h1> I’m going to use “program” for both “programs” and “processes”. There are a million explanations out there about the difference, so I won’t bother here. ↩</a> </li> There are also many built-in commands that have a program equivalent with the same name. For example, echo</code> is a built-in command in bash, but there is also a /usr/bin/echo</code> program. I’m not completely sure why this is the case, but one answer</a> seems to be “POSIX”, a topic I won’t delve into. ↩</a> </li> I’m cheating a bit here. [</code> is actually a shell built-in, but there’s also a program with the same name and functionality, similar to echo</code>. Check the previous footnote for an explanation. ↩</a> </li> </ol> </section>

Understanding gas costs after Berlin

2021-04-11T00:00:00+00:00

Originally published at HackMD</a>.

The Berlin hard fork will go live on mainnet on April 14</a>, introducing four new EIPs. Two of them (EIP-2929</a> and EIP-2930</a>) affect how the gas cost of a transaction is computed. This article explains how (some) gas costs were calculated before Berlin, how this will change with EIP-2929, and how to use the Access Lists feature introduced by EIP-2930.

tl;dr</h2>
This is a long article, so you can just read this and close the tab:

The Berlin hard fork changes the gas cost of some opcodes. If you have a hardcoded gas value in a dapp or a smart contract, they might stop working. And if this happens and the smart contract is non-upgradeable, consumers will need to employ access lists (EIP-2930) to use it.</li>
Access lists can be used to reduce gas costs a little, but they could actually increase the total gas consumed in some scenarios.</li>
geth</code> includes a new RPC method called eth_createAccessList</code> to simplify the creation of access lists.</li> </ul> Gas costs before Berlin</h2> Each opcode executed by the EVM has an associated gas cost. For most of them, this cost is fixed: PUSH1</code> always consumes 3 units of gas, MUL</code> consumes 5, and so on. For others it is variable: the SHA3</code> opcode’s cost depends on the size of its input, for example. We’ll focus on the SLOAD</code> and SSTORE</code> opcodes since they are the ones most affected by the Berlin hard fork. We’ll talk later about those that target an address, like all the EXT*</code> and CALL*</code> opcodes, since their gas cost changes too. SLOAD</code> before Berlin</h3> Without EIP-2929, the cost of SLOAD</code> is simple: it always costs 800 of gas. So there isn’t much to say about it (for now). SSTORE</code> before Berlin</h3> SSTORE</code> is maybe the most complex opcode in terms of gas because its cost depends on things like the current value of the storage slot, the new value, and whether it was previously modified. We’ll analyze only some scenarios to get a basic understanding; if you want to learn more, read the EIPs linked at the end of this article. If the value of the slot changes from 0 to 1 (or any non-zero value), the cost is 20000</li> If the value of the slot changes from 1 to 2 (or any other non-zero value), the cost is 5000.</li> If the value of the slot changes from 1 (or any non-zero value) to 0, the cost is also 5000, but you get a gas refund at the end of the transaction. We won’t cover refunds in this article since they are not affected by Berlin.</li> If the value was previously modified during the same transaction, all the subsequent SSTORE</code>s cost 800.</li> </ul> The details are not that interesting here. The important part is that SSTORE</code> is expensive and that its cost depends on several factors. Gas costs after EIP-2929</h2> EIP-2929 changes all these values. But before getting into that, we need to talk about an important concept introduced by the EIP: accessed addresses and accessed storage keys. An address or a storage key is considered accessed if it was previously “used” during the transaction. For example, when you CALL</code> another contract, the address of that contract is marked as accessed. Similarly, when you SLOAD</code> or SSTORE</code> some slot, it will be considered accessed for the rest of the transaction. It doesn’t matter which opcode did it: if an SLOAD</code> reads a slot, it will be considered accessed for both the next SLOAD</code>s and the next SSTORE</code>s. Something important to notice here is that storage keys are “inside” some address. As the EIP explains it: When executing a transaction, maintain a set accessed_addresses: Set[Address]</code> and accessed_storage_keys: Set[Tuple[Address, Bytes32]]</code> </blockquote> That is, when we say that a storage slot is accessed, we are actually saying that a pair (address, storageKey)</code> was accessed. With that being said, let’s talk about the new gas costs. SLOAD</code> after Berlin</h3> Before Berlin, SLOAD</code> had a fixed cost of 800. Now, it depends on whether that storage slot was already accessed or not. If it wasn’t accessed, the cost is 2100; if it was, it is 100. So, an SLOAD</code> costs 2000 less if the slot is in the list of accessed storage keys. SSTORE</code> after Berlin</h3> Let’s revisit our previous SSTORE</code> examples in the context of EIP-2929: If the value of the slot changes from 0 to 1 (or any non-zero value), the cost is: 22100 if the storage key wasn’t accessed</li> 20000 if it was</li> </ul> </li> If the value of the slot changes from 1 to 2 (or any other non-zero value), the cost is: 5000 if the storage key wasn’t accessed</li> 2900 if it was</li> </ul> </li> If the value of the slot changes from 1 (or any non-zero value) to 0, the cost is the same as the previous item, plus the refund.</li> If the value was previously modified during the same transaction, all the subsequent SSTORE</code>s cost 100.</li> </ul> As you can see, the first SSTORE</code> costs 2100 less if the slot it’s modifying was previously accessed. Putting it all together</h2> That was a mouthful, so here’s a table comparing all the values mentioned so far: OPCODE</td> Before Berlin</td> After Berlin</td> </tr> Not accessed</td> Accessed</td> </tr> </thead> SLOAD</td> 800</td> 2100</td> 100</td> </tr> SSTORE from 0 to 1</td> 20000</td> 22100</td> 20000</td> </tr> SSTORE from 1 to 2</td> 5000</td> 5000</td> 2900</td> </tr> SLOAD + SSTORE*</td> 5800</td> 5000</td> 3000</td> </tr> SSTORE* + SLOAD</td> 5800</td> 5100</td> 3000</td> </tr> SSTORE of an already written slot</td> 800</td> 100</td> </tr> *From a non-zero value to a different non-zero value, like in the third row </td> </tr> </tbody> </table> </div> Notice that in the last row it doesn’t make sense to talk about the slot having been accessed or not because, if it was previously written, then it was also accessed. EIP-2930: Optional Access List transactions</h2> The other EIP we mentioned at the beginning was EIP-2930. This EIP adds a new type of transaction that can include an access list in the transaction payload. This means that you can declare beforehand which addresses and slots should be considered as accessed before the transaction’s execution starts. For example, an SLOAD</code> of a non-accessed slot costs 2100, but if that slot was included in the transaction’s access list, then that same opcode will cost 100. But if the gas costs are lower when an address or storage key is already accessed, does this mean that we can add everything to the transaction’s access list and get a gas reduction? Yay, free gas! Well, not exactly, because you also need to pay gas for each address and each storage key that you add. Let’s see an example. Say we are sending a transaction to contract A</code>. An access list could look like this: accessList: [{ address: "<address of A>", storageKeys: [ "0x0000000000000000000000000000000000000000000000000000000000000000" ] }] </code></pre> If we send a transaction with this access list, and the first opcode that uses the 0x0</code> slot is a SLOAD</code>, it will cost 100 instead of 2100. That’s a gas reduction of 2000. But each storage key included in the transaction’s access list has a cost of 1900. So we only save 100 of gas. (If the first opcode to access that slot is an SSTORE</code> instead, we would save 2100 of gas, which means that we’d save 200 of gas in total if we consider the cost of the storage key.) Does this mean that we always save gas when using transaction’s with access lists? Well, no, because we also pay gas for the address in the access list ("<address of A>"</code> in our example). Accessed addresses</h2> So far, we’ve been talking only about the SLOAD</code> and SSTORE</code> opcodes, but those aren’t the only ones that change after Berlin. For example, the CALL</code> opcode had a fixed cost of 700. But after EIP-2929 its cost is 2600 if the address is not in the access list and 100 if it is. And, like the accessed storage keys, it doesn’t matter what OPCODE accessed that address before (for example, if an EXTCODESIZE</code> was called first, then that opcode will cost 2600, and any subsequent EXTCODESIZE</code>, CALL</code>, STATICCALL</code> that uses the same address will cost 100). How is this affected by transactions with access lists? For example, if we send a transaction to contract A</code>, and that contract calls another contract B</code>, then we can include an access list like this: accessList: [{ address: "<address of B>", storageKeys: [] }] </code></pre> We’ll have to pay a cost of 2400 to include this access list in the transaction, but then the first opcode that uses the address of B</code> will cost 100 instead of 2600. So we saved 100 of gas by doing this. And if B</code> uses its storage somehow and we know which keys it will use, then we can also include them in the access list and save 100/200 of gas for each one (depending on whether the first opcode is an SLOAD</code> or an SSTORE</code>). But why are we talking about another contract? What happens with the contract that we are calling? Why don’t we do this? accessList: [ {address: "<address of A>", storageKeys: []}, {address: "<address of B>", storageKeys: []}, ] </code></pre> We could do it, but it wouldn’t be worth it because EIP-2929 specifies that the address of the contract that is being called (that is, tx.to</code>) is always included in the accessed_addresses</code> list. So we are paying 2400 more for nothing. Let’s analyze our example of the previous section again: accessList: [{ address: "<address of A>", storageKeys: [ "0x0000000000000000000000000000000000000000000000000000000000000000" ] }] </code></pre> This will actually be wasteful unless we include several storage keys more. If we assume that a storage key is always used first by an SLOAD</code>, then we need at least 24 storage keys just to break even. As you can imagine, analyzing this and creating an access list by hand is not fun. Luckily, there is a better way. The eth_createAccessList</code> RPC method</h2> Geth (starting from version 1.10.2) includes a new eth_createAccessList</code> RPC method that you can use to generate access lists. It is used like eth_estimateGas</code>, but instead of a gas estimation, it returns something like this: { "accessList": [ { "address": "0xb0ee076d7779a6ce152283f009f4c32b5f88756c", "storageKeys": [ "0x0000000000000000000000000000000000000000000000000000000000000000", "0x0000000000000000000000000000000000000000000000000000000000000001" ] } ], "gasUsed": "0x8496" } </code></pre> That is, it gives you the list of addresses and storage keys that will be used by that transaction, plus the gas consumed if the access list is included. (And, like eth_estimateGas</code>, this is an estimation; the list could change when the transaction is actually mined.) But, again, this doesn’t mean that this gas will be lower than the gas used if you just send the same transaction without an access list! I suppose we’ll discover over time what’s the proper way of doing this, but my pseudo-code guess is this: let gasEstimation = estimateGas(tx) let { accessList, gasUsed } = createAccessList(tx) if (gasUsed > gasEstimation) { delete accessList[tx.to] } tx.accessList = accessList; sendTransaction(tx) </code></pre> Unbricking contracts</h2> It is important to mention that the main purpose of Access Lists is not to use gas. As the EIP explains it</a>: Mitigates contract breakage risks introduced by EIP-2929, as transactions could pre-specify and pre-pay for the accounts and storage slots that the transaction plans to access; as a result, in the actual execution, the SLOAD and EXT* opcodes would only cost 100 gas: low enough that it would not only prevent breakage due to that EIP but also “unstuck” any contracts that became stuck due to EIP 1884. </blockquote> This means that if a contract makes assumptions about the cost of executing something, the increase in gas costs could break it. For example, a contract makes a call to another contract like someOtherContract.someFunction{gas: 34500}()</code> because it assumes that someFunction</code> will use exactly 34500 of gas, then it will break. But if you include the proper access list in the transaction, then the contract will work again. Check it yourself</h2> If you want to test this yourself, clone this repo</a> which has several examples that can be executed using Hardhat</a> and geth. Check the README for the instructions. Was this fun?</h2> If this low-level mumbo-jumbo is your idea of fun, you should know that Nomic Labs</a> is hiring</a>. References</h2> EIP-2929</a> and EIP-2930</a> are the two Berlin EIPs relevant to this article.</li> EIP-2930 depends on another part of Berlin: EIP-2718</a>, also called Typed Transactions.</li> EIP-2929 refers to EIP-2200</a> a lot, so if you want to understand gas costs more in depth, you should start there.</li> For more complex examples comparing how gas usage changes, check this</a>.</li> </ul>

Examples!</h1>
Ok, now for the fun part. Let’s use what we learned to explore some interesting examples.</p>

Was this fun?</h2>
If this low-level mumbo-jumbo is your idea of fun, you should know that Nomic Labs</a> is hiring</a>.</p>

Whales are Fish

An underqualified reading list about the transformer architecture

You and Your Shell

Hardhat: networks and providers

Understanding gas costs after Berlin

Whales are Fish

An underqualified reading list about the transformer architecture

Closing words</h1> What’s next? I don’t know! As I said in the introduction, I haven’t read much more on the topic than what I shared. But, true to the spirit of talking without knowing, here are some possible next steps:</p>

You and Your Shell

Is this for you?</h1> I think you won’t get much from this article if you can answer these questions without even blinking:</p>

The worst shell in the world</h2> Since we defined a shell as a program that interprets requests to run programs and assumed we have a runProgram</code> function, we can write an extremely simple shell that just runs whatever program is passed to it:</p>

Finding programs</h2> When you want to run the df</code> program to see if you have enough space to keep downloading</del> buying movies, you don’t write /usr/bin/df</code>, you just write df</code>. This wouldn’t work on our dumb implementation, but it does in any real shell.</p>

Examples!</h1> Ok, now for the fun part. Let’s use what we learned to explore some interesting examples.</p>

Hardhat: networks and providers

Understanding gas costs after Berlin

Examples!</h1>
Ok, now for the fun part. Let’s use what we learned to explore some interesting examples.</p>