RustArena

Step 4 of 7

Structs that hold references

A struct can hold a reference instead of owning its data. That's how fast parsers work: instead of copying every piece of the input into new Strings, they hand out slices of the original text. But a struct holding a reference must not outlive the data it points at, so the struct gets a lifetime parameter:

struct Tokenizer {
    rest: &str,
}

fn main() {}

error[E0106]: missing lifetime specifier

With the lifetime declared, here's a small tokenizer that splits a command into words, without copying a single byte:

struct Tokenizer<'a> {
    rest: &'a str,
}

impl<'a> Tokenizer<'a> {
    fn new(input: &'a str) -> Self {
        Tokenizer { rest: input }
    }

    fn next_token(&mut self) -> Option<&'a str> {
        let trimmed = self.rest.trim_start();
        if trimmed.is_empty() {
            return None;
        }
        let end = trimmed.find(' ').unwrap_or(trimmed.len());
        let (token, rest) = trimmed.split_at(end);
        self.rest = rest;
        Some(token)
    }
}

fn main() {
    let command = String::from("move  north 3");
    let mut t = Tokenizer::new(&command);
    let mut tokens = Vec::new();
    while let Some(tok) = t.next_token() {
        tokens.push(tok);
    }
    println!("{:?}", tokens);
}

["move", "north", "3"]

Piece by piece:

Why &'a str and not just &str?

With -> Option<&str>, elision rule 3 would tie each token to &mut self: the tokenizer would stay mutably borrowed for as long as the token is used. Collecting tokens in a loop would then fail, because the next call needs &mut t again while the earlier tokens (in tokens) still hold the borrow:

error[E0499]: cannot borrow `t` as mutable more than once at a time
   |
   |     while let Some(tok) = t.next_token() {
   |                           ^ `t` was mutably borrowed here in the previous iteration of the loop
   |         tokens.push(tok);
   |         ------ first borrow used here, in later iteration of loop

Writing 'a says the truth: a token is a slice of the original command, and stays valid as long as the command does, no matter what happens to the tokenizer.

The struct can't outlive its data

struct Tokenizer<'a> {
    rest: &'a str,
}

fn main() {
    let t;
    {
        let cmd = String::from("go north");
        t = Tokenizer { rest: &cmd };
    }
    println!("{}", t.rest);
}

error[E0597]: `cmd` does not live long enough

Same error as in step 1, same fixes.

Returning a struct with a lifetime

A function that returns a borrowing struct can use elision, but write '_ so the reader sees that something is borrowed. Since Rust 1.89 the compiler warns if you hide it (fn parse(line: &str) -> Entry gives warning: hiding a lifetime that's elided elsewhere is confusing):

fn parse(line: &str) -> Entry<'_> { ... }          // elided, but visible
fn parse<'a>(line: &'a str) -> Entry<'a> { ... }   // the same thing, spelled out

Common mistakes

Workplace note

Borrowing structs are fast, but their lifetime spreads: anything that holds an Entry<'a> needs a 'a too. Teams typically use them for short-lived views (parsers, iterators, a function that analyzes a buffer) and use owned String fields for data that's stored, returned through many layers, or sent to other threads. The last steps of this module come back to that choice.

Your task

Log lines look like [WARN] disk almost full. Give LogLine a lifetime so its fields can borrow from the line, and write parse_log: it returns Some(LogLine) with the text between the brackets as level and the rest (trimmed) as message, or None if the line doesn't start with [ or has no ].

line.strip_prefix('[') returns the rest of the line after the [, or None if it doesn't start with one. Inside a function returning Option, ? on an Option returns None early.

Challenge 2

Settings come as one string like "host=example.com; port=8080". Write the method get on KeyValues: it splits the input on ;, splits each pair at =, and returns the trimmed value for key (comparing the trimmed key), or None.

The value must borrow from the input ('a), not from the KeyValues: a hidden test drops the KeyValues and keeps using the value. Change the return type accordingly.

Challenge 3

Write a whole program. The first line of stdin is a city; every following line is name,city. Read all of stdin into one String, parse the lines into a Vec<Person> where struct Person<'a> { name: &'a str, city: &'a str } borrows (trimmed) slices of the input, and print the name of everyone in that city, one per line, followed by <n> found. Don't copy any names or cities into new Strings.

Previous: The elision rules Next: 'static: references that live forever