11 Responses to “US-ASCII Issues Redux”

1. yawnmoth Says:
   August 29th, 2006 at 4:28 pm

   Correct me if I’m wrong, but there are only 256 US-ASCII characters, aren’t there? As such, 316 isn’t its own character - it’s two. chr(1) and chr(60). 380 is chr(1) and chr(124), 444 is chr(1).chr(188), etc.

   Of those, 316 and 444 aren’t at all surprising. 380 kinda is, though.

2. RSnake Says:
   August 29th, 2006 at 5:13 pm

   That’s probably what the intention was, but when you output a long width character in that encoding method it works. Like so:

   IMG src="" onerror=alert(65532)>

3. Dean Brettle Says:
   August 29th, 2006 at 7:59 pm

   But a filter that prevents these attacks according to Kurt Huwig’s suggested fix should be clearing the high bit of each *byte* before looking for vectors. It wouldn’t make any sense to look at each long character, right?

   That said, I agree with yawnmoth that chr(1) chr(124) getting interpreted as an angle bracket is definitely interesting since the suggested fix would not have blocked it. Are there any other sequences of 7-bit chars that have that property?

4. RSnake Says:
   August 30th, 2006 at 8:41 am

   Dean, I hadn’t heard that fix… but what do you mean look at the high bit, exactly? Do you mean if the first bit of the char is greater than 7 (as in 7f) then ignore it? Well what if I have 32060 (7D3C in hex) the first bit is not greater than 7. Or am I misunderstanding?

5. yawnmoth Says:
   August 30th, 2006 at 9:27 am

   The idea behind the fix, I think, is that if you do an ‘and’ with 0×7f on every character that characters like chr(ord(’

6. yawnmoth Says:
   August 30th, 2006 at 9:32 am

   Hmmm… WordPress’s filtering out what it thinks are tags broke my post…

   I’ll just use ‘i’, instead.

   If you do an ‘and’ with 0×7f on every character like chr(ord(’i') | 0×80), you won’t have two characters that represent an angle bracket - just one. ie. chr(ord(’i') | 0×80) & chr(0×7f) == ‘i’, etc.

7. Dean Brettle Says:
   August 30th, 2006 at 3:36 pm

   RSnake, the suggested fix is to clear the high bit of each *byte* (not each character) before filtering. For 7D3C, no bits would be cleared but a filter looking for “

8. Dean Brettle Says:
   August 30th, 2006 at 3:38 pm

   [Trying again…]

   RSnake, the suggested fix is to clear the high bit of each *byte* (not each character) before filtering. For 7D3C, no bits would be cleared but a filter looking for a left angle bracket would catch it because 3C is a left angle bracket. As a result, I’m saying that 7D3C isn’t really any more interesting than say 203C (space followed by left angle bracket). Both would be caught by a filter operating on bytes, and any filter operating on text in the US-ACII encoding should clearly be operating on bytes.

   017C (decimal 380) is interesting because it doesn’t contain a left angle bracket. As a result, a filter would probably miss it.

9. cea Says:
   August 31st, 2006 at 4:53 am

   Well, all these characters, when represented in UTF-8 contain either 0xbc or 0×3c which are equivalent to ‘

10. cea Says:
    August 31st, 2006 at 4:55 am

    Well, the previous posting is broken by the use of ‘<’ but the conclusion is that there are no hidden surprises.

11. RSnake Says:
    August 31st, 2006 at 8:40 am

    Amit and I discussed this as well, and I think I was falsely finding issues that weren’t there. Here is his diagnosis:

    I’m afraid the picture is more complicated. Your test
    ended up (I believe) in writing the Unicode symbol
    (e.g. 65532) as DOUBLE UTF-8 encoded. That is, the
    symbol was first encoded in UTF-8 into the 3 bytes EF
    BF BC, and then was AGAIN UTF-8 encoded (each byte
    separately) into C3 AF C2 BF C2 BC, which is what I
    see on the page.

    Now, the browser is instructed to interpret this as
    US-ASCII, meaning, each byte to its own. And the
    terminating byte BC is indeed interpreted as a left
    bracket, hence the desired effect.

    But I claim that:

    1. What you have here is pretty non-standard
    situation, wherein data is DOUBLE encoded. I think in
    typical situations, Unicode data is serialized (e.g.
    by Java) to UTF-8 once.

    2. The issue here is not about “characters”, but
    rather, with their wire representation. As it happens,
    UTF-8 is typically used to represent Unicode
    characters, so the question is simply - which Unicode
    characters have UTF-8 representation whose last byte
    is BC (it’s easy to see that no Unicode symbol has
    UTF-8 representation whose last byte is 3C except
    U+003C - the true open bracket). So for example, 188,
    which is U+00BC, is UTF-8 encoded into C2 BC (bingo).

    Likewise, 316, which is U+013C, is UTF-8 encoded into
    C4 BC (bingo, again…). What you need is to study the
    UTF-8 conversion algorithm, and easily derive the list
    of characters (again, up to issue #1 above, which is
    the strange double encoding you have). I think the net effect of what you’ve
    demonstrated
    is that it’s possible to generate the byte BC in various ways in the HTML
    output.

    -Amit

    What’s confusing is that there actually are variable widths that modify the next character (I see the effect quite a lot now that I’m working with Cheng Peng Su and modifying the fuzzer to use that information). Using bvi, I was able to modify the tests in memory, and sure enough I can see the effect with anything I type:

    55 02 BC -> works
    09 BC -> works

    Etc… As long as that last bit is BC it functions. So the serialization of the various characters strung together is having no effect on the last bit. Anyway, as a result this has uncovered a pretty major flaw in my fuzzer design. Don’t expect it anytime soon… I have to re-think some key issues.