bytestring-0.11.5.2: Fast, compact, strict and lazy byte strings with a list interface

Copyright | (c) Don Stewart 2006-2008 (c) Duncan Coutts 2006-2012
License | BSD-style
Maintainer | dons00@gmail.com, duncan@community.haskell.org
Stability | unstable
Portability | non-portable
Safe Haskell | Safe-Inferred
Language | Haskell2010

Data.ByteString.Internal

Contents

- The `ByteString` type and representation
- Internal indexing
- Conversion with lists: packing and unpacking
- Low level imperative construction
- Conversion to and from ForeignPtrs
- Utilities
- Standard C Functions
- cbits functions
- Chars
- Deprecated and unmentionable
- Exported compatibility shim

Description

A module containing semi-public `ByteString` internals. This exposes the `ByteString` representation and low level construction functions. As such all the functions in this module are unsafe. The API is also not stable.

Where possible application should instead use the functions from the normal public interface modules, such as Data.ByteString.Unsafe. Packages that extend the ByteString system at a low level will need to use this module.

Synopsis

- data ByteString where
  - BS !(ForeignPtr Word8) !Int
  - pattern PS :: ForeignPtr Word8 -> Int -> Int -> ByteString
- type StrictByteString = ByteString
- findIndexOrLength :: (Word8 -> Bool) -> ByteString -> Int
- packBytes :: [Word8] -> ByteString
- packUptoLenBytes :: Int -> [Word8] -> (ByteString, [Word8])
- unsafePackLenBytes :: Int -> [Word8] -> ByteString
- packChars :: [Char] -> ByteString
- packUptoLenChars :: Int -> [Char] -> (ByteString, [Char])
- unsafePackLenChars :: Int -> [Char] -> ByteString
- unpackBytes :: ByteString -> [Word8]
- unpackAppendBytesLazy :: ByteString -> [Word8] -> [Word8]
- unpackAppendBytesStrict :: ByteString -> [Word8] -> [Word8]
- unpackChars :: ByteString -> [Char]
- unpackAppendCharsLazy :: ByteString -> [Char] -> [Char]
- unpackAppendCharsStrict :: ByteString -> [Char] -> [Char]
- unsafePackAddress :: Addr# -> IO ByteString
- unsafePackLenAddress :: Int -> Addr# -> IO ByteString
- unsafePackLiteral :: Addr# -> ByteString
- unsafePackLenLiteral :: Int -> Addr# -> ByteString
- empty :: ByteString
- create :: Int -> (Ptr Word8 -> IO ()) -> IO ByteString
- createUptoN :: Int -> (Ptr Word8 -> IO Int) -> IO ByteString
- createUptoN' :: Int -> (Ptr Word8 -> IO (Int, a)) -> IO (ByteString, a)
- createAndTrim :: Int -> (Ptr Word8 -> IO Int) -> IO ByteString
- createAndTrim' :: Int -> (Ptr Word8 -> IO (Int, Int, a)) -> IO (ByteString, a)
- unsafeCreate :: Int -> (Ptr Word8 -> IO ()) -> ByteString
- unsafeCreateUptoN :: Int -> (Ptr Word8 -> IO Int) -> ByteString
- unsafeCreateUptoN' :: Int -> (Ptr Word8 -> IO (Int, a)) -> (ByteString, a)
- mallocByteString :: Int -> IO (ForeignPtr a)
- mkDeferredByteString :: ForeignPtr Word8 -> Int -> IO ByteString
- fromForeignPtr :: ForeignPtr Word8 -> Int -> Int -> ByteString
- toForeignPtr :: ByteString -> (ForeignPtr Word8, Int, Int)
- fromForeignPtr0 :: ForeignPtr Word8 -> Int -> ByteString
- toForeignPtr0 :: ByteString -> (ForeignPtr Word8, Int)
- nullForeignPtr :: ForeignPtr Word8
- deferForeignPtrAvailability :: ForeignPtr a -> IO (ForeignPtr a)
- checkedAdd :: String -> Int -> Int -> Int
- c_strlen :: CString -> IO CSize
- c_free_finalizer :: FunPtr (Ptr Word8 -> IO ())
- memchr :: Ptr Word8 -> Word8 -> CSize -> IO (Ptr Word8)
- memcmp :: Ptr Word8 -> Ptr Word8 -> Int -> IO CInt
- memcpy :: Ptr Word8 -> Ptr Word8 -> Int -> IO ()
- memset :: Ptr Word8 -> Word8 -> CSize -> IO (Ptr Word8)
- c_reverse :: Ptr Word8 -> Ptr Word8 -> CSize -> IO ()
- c_intersperse :: Ptr Word8 -> Ptr Word8 -> CSize -> Word8 -> IO ()
- c_maximum :: Ptr Word8 -> CSize -> IO Word8
- c_minimum :: Ptr Word8 -> CSize -> IO Word8
- c_count :: Ptr Word8 -> CSize -> Word8 -> IO CSize
- c_sort :: Ptr Word8 -> CSize -> IO ()
- w2c :: Word8 -> Char
- c2w :: Char -> Word8
- isSpaceWord8 :: Word8 -> Bool
- isSpaceChar8 :: Char -> Bool
- accursedUnutterablePerformIO :: IO a -> a
- plusForeignPtr :: ForeignPtr a -> Int -> ForeignPtr b
- unsafeWithForeignPtr :: ForeignPtr a -> (Ptr a -> IO b) -> IO b

# The `ByteString` type and representation

data ByteString Source #

A space-efficient representation of a `Word8` vector, supporting many efficient operations.

A `ByteString` contains 8-bit bytes, or by using the operations from Data.ByteString.Char8 it can be interpreted as containing 8-bit characters.

Constructors

BS !(ForeignPtr Word8) !Int | Since: bytestring-0.11.0.0

Bundled Patterns

pattern PS :: ForeignPtr Word8 -> Int -> Int -> ByteString

``PS` foreignPtr offset length` represents a `ByteString` with data backed by a given `foreignPtr`, starting at a given `offset` in bytes and of a specified `length`.

This pattern is used to emulate the legacy `ByteString` data constructor, so that pre-existing code generally doesn't need to change to benefit from the simplified `BS` constructor and can continue to function unchanged.

Note: Matching with this constructor will always be given a 0 offset, as the base will be manipulated by `plusForeignPtr` instead.

#### Instances

Instances details

Data ByteString Source # |

Instance details

Defined in Data.ByteString.Internal.Type

Methods

gfoldl :: (forall d b. Data d => c (d -> b) -> d -> c b) -> (forall g. g -> c g) -> ByteString -> c ByteString Source #

gunfold :: (forall b r. Data b => c (b -> r) -> c r) -> (forall r. r -> c r) -> Constr -> c ByteString Source #

toConstr :: ByteString -> Constr Source #

dataTypeOf :: ByteString -> DataType Source #

dataCast1 :: Typeable t => (forall d. Data d => c (t d)) -> Maybe (c ByteString) Source #

dataCast2 :: Typeable t => (forall d e. (Data d, Data e) => c (t d e)) -> Maybe (c ByteString) Source #

gmapT :: (forall b. Data b => b -> b) -> ByteString -> ByteString Source #

gmapQl :: (r -> r' -> r) -> r -> (forall d. Data d => d -> r') -> ByteString -> r Source #

gmapQr :: forall r r'. (r' -> r -> r) -> r -> (forall d. Data d => d -> r') -> ByteString -> r Source #

gmapQ :: (forall d. Data d => d -> u) -> ByteString -> [u] Source #

gmapQi :: Int -> (forall d. Data d => d -> u) -> ByteString -> u Source #

gmapM :: Monad m => (forall d. Data d => d -> m d) -> ByteString -> m ByteString Source #

gmapMp :: MonadPlus m => (forall d. Data d => d -> m d) -> ByteString -> m ByteString Source #

gmapMo :: MonadPlus m => (forall d. Data d => d -> m d) -> ByteString -> m ByteString Source #

IsString ByteString Source # | Beware: `fromString` truncates multi-byte characters to octets. e.g. "枯朶に烏のとまりけり秋の暮" becomes �6k�nh~�Q��n�
Instance details Defined in Data.ByteString.Internal.Type Methods fromString :: String -> ByteString Source #
Monoid ByteString Source # |
Instance details Defined in Data.ByteString.Internal.Type Methods mempty :: ByteString Source # mappend :: ByteString -> ByteString -> ByteString Source # mconcat :: [ByteString] -> ByteString Source #
Semigroup ByteString Source # |
Instance details Defined in Data.ByteString.Internal.Type Methods (<>) :: ByteString -> ByteString -> ByteString Source # sconcat :: NonEmpty ByteString -> ByteString Source # stimes :: Integral b => b -> ByteString -> ByteString Source #
IsList ByteString Source # | Since: bytestring-0.10.12.0
Instance details Defined in Data.ByteString.Internal.Type Associated Types type Item ByteString Source # Methods fromList :: [Item ByteString] -> ByteString Source # fromListN :: Int -> [Item ByteString] -> ByteString Source # toList :: ByteString -> [Item ByteString] Source #
Read ByteString Source # |
Instance details Defined in Data.ByteString.Internal.Type Methods readsPrec :: Int -> ReadS ByteString Source # readList :: ReadS [ByteString] Source # readPrec :: ReadPrec ByteString Source # readListPrec :: ReadPrec [ByteString] Source #
Show ByteString Source # |
Instance details Defined in Data.ByteString.Internal.Type Methods showsPrec :: Int -> ByteString -> ShowS Source # show :: ByteString -> String Source # showList :: [ByteString] -> ShowS Source #
NFData ByteString Source # |
Instance details Defined in Data.ByteString.Internal.Type Methods rnf :: ByteString -> () Source #
Eq ByteString Source # |
Instance details Defined in Data.ByteString.Internal.Type Methods (==) :: ByteString -> ByteString -> Bool Source # (/=) :: ByteString -> ByteString -> Bool Source #
Ord ByteString Source # |

Instance details

Defined in Data.ByteString.Internal.Type

Methods

compare :: ByteString -> ByteString -> Ordering Source #

(<) :: ByteString -> ByteString -> Bool Source #

(<=) :: ByteString -> ByteString -> Bool Source #

(>) :: ByteString -> ByteString -> Bool Source #

(>=) :: ByteString -> ByteString -> Bool Source #

max :: ByteString -> ByteString -> ByteString Source #

min :: ByteString -> ByteString -> ByteString Source #

Lift ByteString Source # | Since: bytestring-0.11.2.0
Instance details Defined in Data.ByteString.Internal.Type Methods lift :: Quote m => ByteString -> m Exp Source # liftTyped :: forall (m :: Type -> Type). Quote m => ByteString -> Code m ByteString Source #
type Item ByteString Source # |
Instance details Defined in Data.ByteString.Internal.Type type Item ByteString = Word8

type StrictByteString = ByteString Source #

Type synonym for the strict flavour of `ByteString`.

Since: bytestring-0.11.2.0

# Internal indexing

findIndexOrLength :: (Word8 -> Bool) -> ByteString -> Int Source #

`findIndexOrLength` is a variant of findIndex, that returns the length of the string if no element is found, rather than Nothing.

# Conversion with lists: packing and unpacking

packBytes :: [Word8] -> ByteString Source #

packUptoLenBytes :: Int -> [Word8] -> (ByteString, [Word8]) Source #

unsafePackLenBytes :: Int -> [Word8] -> ByteString Source #

packChars :: [Char] -> ByteString Source #

packUptoLenChars :: Int -> [Char] -> (ByteString, [Char]) Source #

unsafePackLenChars :: Int -> [Char] -> ByteString Source #

unpackBytes :: ByteString -> [Word8] Source #

unpackAppendBytesLazy :: ByteString -> [Word8] -> [Word8] Source #

unpackAppendBytesStrict :: ByteString -> [Word8] -> [Word8] Source #

unpackChars :: ByteString -> [Char] Source #

unpackAppendCharsLazy :: ByteString -> [Char] -> [Char] Source #

unpackAppendCharsStrict :: ByteString -> [Char] -> [Char] Source #

unsafePackAddress :: Addr# -> IO ByteString Source #

O(n) Pack a null-terminated sequence of bytes, pointed to by an Addr# (an arbitrary machine address assumed to point outside the garbage-collected heap) into a `ByteString`. A much faster way to create an `Addr#` is with an unboxed string literal, than to pack a boxed string. A unboxed string literal is compiled to a static `char []` by GHC. Establishing the length of the string requires a call to `strlen(3)`, so the `Addr#` must point to a null-terminated buffer (as is the case with `"string"#` literals in GHC). Use `unsafePackAddressLen` if you know the length of the string statically.

An example:

    literalFS = unsafePackAddress "literal"#

This function is unsafe. If you modify the buffer pointed to by the original `Addr#` this modification will be reflected in the resulting `ByteString`, breaking referential transparency.

Note this also won't work if your `Addr#` has embedded `'\0'` characters in the string, as `strlen` will return too short a length.

unsafePackLenAddress :: Int -> Addr# -> IO ByteString Source #

See `unsafePackAddress`. This function is similar, but takes an additional length argument rather then computing it with `strlen`. Therefore embedding `'\0'` characters is possible.

Since: bytestring-0.11.2.0

unsafePackLiteral :: Addr# -> ByteString Source #

See `unsafePackAddress`. This function has similar behavior. Prefer this function when the address in known to be an `Addr#` literal. In that context, there is no need for the sequencing guarantees that `IO` provides. On GHC 9.0 and up, this function uses the `FinalPtr` data constructor for `ForeignPtrContents`.

Since: bytestring-0.11.1.0

unsafePackLenLiteral :: Int -> Addr# -> ByteString Source #

See `unsafePackLiteral`. This function is similar, but takes an additional length argument rather then computing it with `strlen`. Therefore embedding `'\0'` characters is possible.

Since: bytestring-0.11.2.0

# Low level imperative construction

empty :: ByteString Source #

O(1) The empty `ByteString`

create :: Int -> (Ptr Word8 -> IO ()) -> IO ByteString Source #

Create ByteString of size `l` and use action `f` to fill its contents.

createUptoN :: Int -> (Ptr Word8 -> IO Int) -> IO ByteString Source #

Given a maximum size `l` and an action `f` that fills the `ByteString` starting at the given `Ptr` and returns the actual utilized length, ``createUptoN'` l f` returns the filled `ByteString`.

createUptoN' :: Int -> (Ptr Word8 -> IO (Int, a)) -> IO (ByteString, a) Source #

Like `createUptoN`, but also returns an additional value created by the action.

Since: bytestring-0.10.12.0

createAndTrim :: Int -> (Ptr Word8 -> IO Int) -> IO ByteString Source #

Given the maximum size needed and a function to make the contents of a ByteString, createAndTrim makes the `ByteString`. The generating function is required to return the actual final size (<= the maximum size), and the resulting byte array is reallocated to this size.

createAndTrim is the main mechanism for creating custom, efficient ByteString functions, using Haskell or C functions to fill the space.

createAndTrim' :: Int -> (Ptr Word8 -> IO (Int, Int, a)) -> IO (ByteString, a) Source #

unsafeCreate :: Int -> (Ptr Word8 -> IO ()) -> ByteString Source #

A way of creating ByteStrings outside the IO monad. The `Int` argument gives the final size of the ByteString.

unsafeCreateUptoN :: Int -> (Ptr Word8 -> IO Int) -> ByteString Source #

Like `unsafeCreate` but instead of giving the final size of the ByteString, it is just an upper bound. The inner action returns the actual size. Unlike `createAndTrim` the ByteString is not reallocated if the final size is less than the estimated size.

unsafeCreateUptoN' :: Int -> (Ptr Word8 -> IO (Int, a)) -> (ByteString, a) Source #

Since: bytestring-0.10.12.0

mallocByteString :: Int -> IO (ForeignPtr a) Source #

Wrapper of `mallocForeignPtrBytes` with faster implementation for GHC

# Conversion to and from ForeignPtrs

mkDeferredByteString :: ForeignPtr Word8 -> Int -> IO ByteString Source #

Variant of `fromForeignPtr0` that calls `deferForeignPtrAvailability`

Since: bytestring-0.11.5.0

fromForeignPtr Source #

Arguments

:: ForeignPtr Word8 |
-> Int | Offset
-> Int | Length
-> ByteString |

O(1) Build a ByteString from a ForeignPtr.

If you do not need the offset parameter then you should be using `unsafePackCStringLen` or `unsafePackCStringFinalizer` instead.

toForeignPtr Source #

Arguments

:: ByteString |
-> (ForeignPtr Word8, Int, Int) | (ptr, offset, length)

O(1) Deconstruct a ForeignPtr from a ByteString

fromForeignPtr0 Source #

Arguments

:: ForeignPtr Word8 |
-> Int | Length
-> ByteString |

Since: bytestring-0.11.0.0

toForeignPtr0 Source #

Arguments

:: ByteString |
-> (ForeignPtr Word8, Int) | (ptr, length)

O(1) Deconstruct a ForeignPtr from a ByteString

Since: bytestring-0.11.0.0

# Utilities

nullForeignPtr :: ForeignPtr Word8 Source #

The 0 pointer. Used to indicate the empty Bytestring.

deferForeignPtrAvailability :: ForeignPtr a -> IO (ForeignPtr a) Source #

Most operations on a `ByteString` need to read from the buffer given by its `ForeignPtr Word8` field. But since most operations on `ByteString` are (nominally) pure, their implementations cannot see the IO state thread that was used to initialize the contents of that buffer. This means that under some circumstances, these buffer-reads may be executed before the writes used to initialize the buffer are executed, with unpredictable results.

`deferForeignPtrAvailability` exists to help solve this problem. At runtime, a call ``deferForeignPtrAvailability` x` is equivalent to `pure $! x`, but the former is more opaque to the simplifier, so that reads from the pointer in its result cannot be executed until the ``deferForeignPtrAvailability` x` call is complete.

The opaque bits evaporate during CorePrep, so using `deferForeignPtrAvailability` incurs no direct overhead.

Since: bytestring-0.11.5.0

checkedAdd :: String -> Int -> Int -> Int Source #

Add two non-negative numbers. Errors out on overflow. ...

# Standard C Functions

c_strlen :: CString -> IO CSize Source #

c_free_finalizer :: FunPtr (Ptr Word8 -> IO ()) Source #

memchr :: Ptr Word8 -> Word8 -> CSize -> IO (Ptr Word8) Source #

memcmp :: Ptr Word8 -> Ptr Word8 -> Int -> IO CInt Source #

memcpy :: Ptr Word8 -> Ptr Word8 -> Int -> IO () Source #

Deprecated: Use Foreign.Marshal.Utils.copyBytes instead

deprecated since `bytestring-0.11.5.0`

memset :: Ptr Word8 -> Word8 -> CSize -> IO (Ptr Word8) Source #

Deprecated: Use Foreign.Marshal.Utils.fillBytes instead

deprecated since `bytestring-0.11.5.0`

# cbits functions

c_reverse :: Ptr Word8 -> Ptr Word8 -> CSize -> IO () Source #

c_intersperse :: Ptr Word8 -> Ptr Word8 -> CSize -> Word8 -> IO () Source #

c_maximum :: Ptr Word8 -> CSize -> IO Word8 Source #

c_minimum :: Ptr Word8 -> CSize -> IO Word8 Source #

c_count :: Ptr Word8 -> CSize -> Word8 -> IO CSize Source #

c_sort :: Ptr Word8 -> CSize -> IO () Source #

# Chars

w2c :: Word8 -> Char Source #

Conversion between `Word8` and `Char`. Should compile to a no-op.

c2w :: Char -> Word8 Source #

Unsafe conversion between `Char` and `Word8`. This is a no-op and silently truncates to 8 bits Chars > '255'. It is provided as convenience for ByteString construction.

isSpaceWord8 :: Word8 -> Bool Source #

Selects words corresponding to white-space characters in the Latin-1 range

isSpaceChar8 :: Char -> Bool Source #

Selects white-space characters in the Latin-1 range

# Deprecated and unmentionable

accursedUnutterablePerformIO :: IO a -> a Source #

This "function" has a superficial similarity to `unsafePerformIO` but it is in fact a malevolent agent of chaos. It unpicks the seams of reality (and the `IO` monad) so that the normal rules no longer apply. It lulls you into thinking it is reasonable, but when you are not looking it stabs you in the back and aliases all of your mutable buffers. The carcass of many a seasoned Haskell programmer lie strewn at its feet.

Witness the trail of destruction:

- https://github.com/haskell/bytestring/commit/71c4b438c675aa360c79d79acc9a491e7bbc26e7
- https://github.com/haskell/bytestring/commit/210c656390ae617d9ee3b8bcff5c88dd17cef8da
- https://github.com/haskell/aeson/commit/720b857e2e0acf2edc4f5512f2b217a89449a89d
- https://ghc.haskell.org/trac/ghc/ticket/3486
- https://ghc.haskell.org/trac/ghc/ticket/3487
- https://ghc.haskell.org/trac/ghc/ticket/7270
- https://gitlab.haskell.org/ghc/ghc/-/issues/22204

Do not talk about "safe"! You do not know what is safe!

Yield not to its blasphemous call! Flee traveller! Flee or you will be corrupted and devoured!

# Exported compatibility shim

plusForeignPtr :: ForeignPtr a -> Int -> ForeignPtr b Source #

Advances the given address by the given offset in bytes.

The new `ForeignPtr` shares the finalizer of the original, equivalent from a finalization standpoint to just creating another reference to the original. That is, the finalizer will not be called before the new `ForeignPtr` is unreachable, nor will it be called an additional time due to this call, and the finalizer will be called with the same address that it would have had this call not happened, *not* the new address.

Since: base-4.10.0.0

unsafeWithForeignPtr :: ForeignPtr a -> (Ptr a -> IO b) -> IO b Source #

This is similar to `withForeignPtr` but comes with an important caveat: the user must guarantee that the continuation does not diverge (e.g. loop or throw an exception). In exchange for this loss of generality, this function offers the ability of GHC to optimise more aggressively.

Specifically, applications of the form: `unsafeWithForeignPtr fptr (`forever` something)`

See GHC issue #17760 for more information about the unsoundness behavior that this function can result in.