What is the difference between "bracket (mallocBytes n) free" and "allocaBytes"?

Question

If you want background, see here. In short, the question is: “What's actual difference between bracket (mallocBytes n) free and allocaBytes from Foreign.Marshall.Alloc”.

Normally in C, alloca allocates on stack and malloc allocates on heap. I'm not sure what's going on with it in Haskell, but I would not expect a difference between the above-mentioned equations other than speed. If you clicked the background link though, you know that with compiled code bracket (mallocBytes n) free was resulting in “double free or corruption” while allocaBytes works fine (the problem is not visible when in GHCi at all, everything works fine in both cases).

By now I've spent two days in painful debugging and I'm pretty confident that bracket (mallocBytes n) free was unstable somehow and the rest of the code is reliable. I'd like to find out what's the deal with bracket (mallocBytes n) free.

Answer 1

bracket (mallocBytes size) free will use C's malloc and free, whereas allocaBytes size will use memory that's managed by GHCs garbage collection. That in itself is a huge difference already, since the Ptr of allocaBytes might be surrounded by unused (but allocated) memory:

import Control.Exception
import Control.Monad (forM_)
import Foreign.Marshal.Alloc
import Foreign.Ptr
import Foreign.Storable

-- Write a value at an invalid pointer location
hammer :: Ptr Int -> IO ()
hammer ptr = pokeElemOff ptr (-1) 0 >> putStrLn "hammered"

main :: IO ()
main = do
  putStrLn "Hammer time! Alloca!"
  forM_ [1..10] $ \n ->
    print n >> allocaBytes 10 hammer

  putStrLn "Hammer time! Bracket"
  forM_ [1..10] $ \n ->
    print n >> bracket (mallocBytes 10) free hammer

Result:

Hammer time! Alloca!
1
hammered
2
hammered
3
hammered
4
hammered
5
hammered
6
hammered
7
hammered
8
hammered
9
hammered
10
hammered
Hammer time! Bracket
1
hammered
<program crashes>

As you can see, although we've used arr[-1] = 0, allocaBytes happily ignored that error. However, free will (often) blow up in your face if you write to position -1. It will also blow up in your face if there has been a memory corruption on another allocated memory region*.

Also, with allocaBytes, it's likely that the pointer points somewhere into already allocated memory, not to the start of one, e.g.

nursery = malloc(NURSERY_SIZE);

// ...

pointer_for_user = nursery + 180;

// pointer_for_user[-1] = 0 is not as 
// much as a problem, since it doesn't yield undefined behaviour

What does that mean? Well, allocaBytes is less likely to blow up in your face, but at the cost that you don't notice if your C-code variant would lead to memory corruption. Even worse, as soon as you write outside of the bounds returned by allocaBytes, your possibly corrupting other Haskell values silently.

However, we're talking here about undefined behaviour. The code above may or may not crash on your system. It may also crash in the allocaBytes part.

If I were you, I would trace the malloc and free calls.

---

^{* I once had a "double use of free" error in the middle of my program. Debugged everything, rewrote most of the "bad" routine. Unfortunately, the error vanished in debug builds, but recurred in release builds. It turned out that in the first ten lines of main, I accidentally wrote to b[i - 1] with i = 0.}

Answer 2

I was able to duplicate the problem, and I can confirm that there is substantial buffer overrun going on. If you use the following allocator (please excuse the quick and dirty code) which adds a page's worth of 0xa5s after the buffer and dumps it out if it's modified, you can see overrun of several hundred bytes in several tests:

withBuffer :: Int -> (Ptr a -> IO b) -> IO b
withBuffer n = bracket begin end
  where begin = do
          a <- mallocBytes (n + 4096)
          mapM_ (\i -> pokeByteOff (a `plusPtr` n) i (0xa5 :: Word8)) [0..4095]
          return a
        end = \a -> do
          page <- mapM (\i -> peekByteOff (a `plusPtr` n) i) [0..4095]
          when (any (/= (0xa5 :: Word8)) page) $ do
            putStrLn $ unlines $ map (hexline page) [0,16..4095]
            error "corruption detected"
          free a
        hexline bytes off = unwords . map hex . take 16 . drop off $ bytes
        hex = printf "%02x"