Silver Challenge...stuck on the token

I got the invalid character portion of this challenge without any issue, but I’m stuck on the token portion…

catch Parser.Error.invalidToken(let token) {
        let distanceToInvalidToken = input.distance(from: input.startIndex, to: input.firstIndex(of: [character associated to this token])
        // how do I get the original value from input that the token represents?
        print("Invalid token during parsing at index \(distanceToInvalidToken): \(token)")

I don’t have the book, but more information should be available at the point the error is detected. You can take advantage of that and raise an error exception that includes the required information.

The way Token is defined, you can’t be guaranteed to have access to that since it’s conditional. If the token is a .number you can, but if it’s a .plus you cannot. You’d have to do something like this:

var value: String
switch token {
case let .number(digit):
    value = String(digit)
case .plus:
    value = "+"
case .minus:
    value = "-"
}

But the way you’re using that won’t work. Consider the example from the book: “10 + 3 3 + 7”. The invalid token is “3” but it isn’t the first “3” in the input string, so using firstIndex(of:) won’t give you the position of the invalid token. When invalidToken is thrown you’ll need to include the position of the token alongside the token itself, so you’ll need to save that information when the token is created in Lexer.

I’m don’t understand where or how to implement your switch statement.

As for trying to save the information in Lexer, I’ve also tried adding a new variable:

class Lexer {
...
var errorPosition: String.Index
...
init(input: String) {
        self.input = input
        self.position = input.startIndex
        self.errorPosition = input.startIndex
    }

...

func lex() throws -> [Token] {
        var tokens = [Token] ()
        
        while let nextCharacter = peek() {
            switch nextCharacter {
            ...
            default:
                // Something unexpected - need to send back an error
                errorPosition = input.index(after: position) // silver challenge - trying to capture the position of the error character...doesn't position change as you advance() within the other cases?
                throw Lexer.Error.invalidCharacter(nextCharacter)
            }
        }

But with my output, the errorPosition never changes:

Evaluating: 10 + 3 3 + 7
12
Lexer output: [Number: 10, Symbol: +, Number: 3, Number: 3, Symbol: +, Number: 7]
Invalid token during parsing at index 0: Number: 3

You would put it in the code that’s catching the error:

} catch Parser.Error.invalidToken(let token) {
    var badChar: String
    switch token {
    case let .number(digit):
        badChar = String(digit)
    case .plus:
        badChar = "+"
    case .minus:
        badChar = "-"
    }
    let distanceToInvalidToken = input.distance(from: input.startIndex, to: input.firstIndex(of: badChar))
    print("Invalid token during parsing: \(token) at position: \(distanceToInvalidToken)")

But as I said, that doesn’t always find the right character.

As for your new code, it isn’t working because you put it in the wrong place. The error you’re getting is invalidToken which is thrown from parse(); lex() isn’t throwing invalidCharacter because there’s no invalid character in the input string so the code you added to lex() never gets called. lex() needs to save the location of all the tokens, so when parse() throws invalidToken it can look up where the bad token was located.

Thanks for clarifying.

I see what you’re saying about not finding the right character all the time.

This isn’t correct:

Evaluating: 10 + 3 3 + 7
Lexer output: [Number: 10, Symbol: +, Number: 3, Number: 3, Symbol: +, Number: 7]
Invalid token during parsing Number: 3 at index 5.

Though, this is:

Evaluating: 10 + 3 8 + 7
Lexer output: [Number: 10, Symbol: +, Number: 3, Number: 8, Symbol: +, Number: 7]
Invalid token during parsing Number: 8 at index 7.

I also got a compile error with my distanceToInvalidToken stating

rror: cannot convert value of type 'String' to expected argument type 'String.Element' (aka 'Character')

But once I convert it to Character, it crashes when I replace ‘3’ with ‘33’.

I still don’t understand why it’s so difficult to simply capture the exact character’s exact index to calculate an accurate distanceToInvalidToken.

Right, firstIndex(of:) only works on individual characters, you can’t pass in a string. If you want to find the location a token that’s longer than a single character, you’d have to do something like:

let r = input.range(of: badChar)   // where badChar is a String
let distanceToInvalidToken = input.distance(from: input.startIndex, to: r!.lowerBound)

which would give you the location of the first character in the first occurrance of badChar.

As I suggested before, it’s easier to do this calculation inside lex() because it knows which characters it’s reading for each token. You don’t have to search for anything

I’m trying to do this in lex(), though it’s very inefficient…

I’ve brought back my errorPosition variable and am calculating this within each case of my switch nextCharacter

errorPosition = input.distance(from: input.startIndex, to: position)

Now, I can capture the position of the number associated to the token. But now how do I make it available to the token?


Also, was this easier for everyone else on these forums to complete? I can’t be the only one struggling with these challenges.

If it’s easier for you, I’d be interested in learning more from anyone not named @JonAult about how you tackle these more easily than I’m trying to.

Update: I’ve re-read the chapter but am nowhere near the correct solution to this challenge.

This is beyond frustrating.

If you were as frustrated as I am, how did you overcome the frustration to reach the solution?

If this was an easy challenge for you, then why was it easy, and what was your step-by-step approach to reaching the solution?