Dervish Memory Management and Garbage Collection C API


Dervish provides an extensive API for memory management. The following discussions details each API and wherever appropriate, source code examples are included as well.

For an overview of DERVISH Memory Management and Garbage Collection, and the definition of some commonly used terms, press here.

Descriptions of the C API

  • shMalloc
  • shFree
  • shCalloc
  • shRealloc
  • shMemEmptyCB
  • shMemRefCntrIncr
  • shMemRefCntrDecr
  • shIsShMallocPtr
  • shMemSerialCB
  • shMemSerialFreeCB
  • shMemFreeBlocks
  • shMemStatsPrint
  • shMemSerialNumberGet
  • shMemTotalBytesMalloced
  • shMemBytesInUse
  • shMemBytesInPool
  • shMemNumFrees
  • shMemNumMallocs
  • shMemAMPSize
  • p_shMemCheck
  • shMemBlocksizeSet
  • shMemDefragment

  • shMalloc

    Allocate requested bytes of memory, returning address of the starting byte to the user. The returned pointer is guaranteed to be suitably aligned for any type of object. SYNOPSIS: #include "shCGarbage.h" void *shMalloc(const size_t size) size - number of bytes to allocate RETURNS: On success - pointer to a suitably aligned memory area On failure - there is no failure return from this function. It will abort if it cannot allocate requested memory

    shFree

    De-allocate memory block specified in the parameter. This function will abort if:
  • address of the block to be de-allocated has not been allocated by shMalloc(), or
  • attempt is made to shFree() a block that has already been deallocated.
  • If a memory block to be de-allocated is still being referenced, it will not be de-allocated. Rather, the reference counter associated with that block will be decremented. A memory block will only be de-allocated when it's reference count reaches 0, i.e. it is not being referenced at all. SYNOPSIS: #include "shCGarbage.h" void shFree(void *ptr) ptr - Pointer to the memory block to be de-allocated RETURNS: Nothing

    shCalloc

    Allocates space for an array with the number of elements specified by the num_of_elts parameter, where each element is of the size specified by the elt_size parameter. The space is initialized to all-zero-bits, which on most but not all machines is equivalent to assignment to a zero of the appropriate type. SYNOPSIS: #include "shCGarbage.h" void *shCalloc(size_t size) void *shCalloc(size_t num_of_elts,size_t elt_size); num_of_elts - number of elements to allocate elt_size - size of each element RETURNS: On success - pointer to a suitably aligned memory area On failure - there is no failure return from this function. It will abort if it cannot allocate requested memory

    shRealloc

    shRealloc() takes a pointer to the memory block previously allocated by shMalloc() and changes its size while preserving its contents; Although the system realloc may be able to grow memory blocks in place, shRealloc never does. If the new size is less then the original size, contents will be preserved upto the new size only. If the new size is 0 and the original pointer is not NULL, the original pointer is de-allocated and a NULL is returned. If the original pointer is NULL, a new memory block of the requested size is created and returned. SYNOPSIS: #include "shCGarbage.h" void *shRealloc(void *ptr, size_t size) ptr - Original pointer to the original memory area size - new size RETURNS: On success - A pointer to the new memory area On failure - NULL EXAMPLE: void *pMemBlock; pMemBlock = (char *) shMalloc(20); ... /* * Do something with pMemBlock */ ... /* * Now we realized that we need more then 20 bytes, let's re-alloc */ pMemBlock = (char *) shRealloc(pMemBlock, 200); ...

    shMemEmptyCB

    shMemEmptyCB() registers a user-supplied callback function which is called when a request for memory cannot be fulfilled. The signature of the user-supplied callback function should be as follows: void *callBackFunc(size_t nbyte) Unless the callBackFunc() calls exit(2) or abort(2), control will be passed back to the Dervish memory management routines and execution will continue from the point where the callback function was called. If you return, you must return a pointer to the requested number of bytes. If you don't, there is very little chance that your application will continue successfully! SYNOPSIS: #include "shCGarbage.h" void shMemEmptyCB(const unsigned long trigger, void *(*callBackFuncp)(size_t nbyte)) trigger - serial number to trigger the event on callBackFuncp - address of the user-supplied callback function RETURNS: Nothing. For an example of using callbacks, see shMemSerialCB.


    shMemRefCntrIncr

    shMemRefCntrIncr() increments a reference counter associated with the given memory address. This function will abort if the memory address passed as the parameter was not allocated using shMalloc(). SYNOPSIS: #include "shCGarbage.h" void shMemRefCntrIncr(void *addr) addr - Memory address whose reference counter is to be incremented RETURN: Nothing EXAMPLE: char *sp1, *sp2; ... sp1 = shMalloc(20); ... /* * Now we need to set sp2 to sp1, but we want to make sure that if sp1 is * deallocated, sp2 still remains a valid pointer. How do we do that? we * increment sp1's reference counter as follows: */ sp2 = sp1; shMemRefCntrIncr(sp1); ...

    shMemRefCntrDecr

    shMemRefCntrDecr() is the complement of shMemRefCntrIncr(). It decrements a reference counter associated with the memory address passed in as the parameter. The memory block will be de-allocated only when the reference counter reaches 0. #include "shCGarbage.h" void shMemCntrDecr(void *addr) addr - Memory address whose reference counter is to be decremented RETURNS: Nothing EXAMPLE: char *sp1, *sp2; ... sp1 = shMalloc(20); ... sp2 = sp1; shMemRefCntrIncr(sp1); ... shFree(sp1); /* This will not deallocate block pointed to by sp1, since it is still being referenced by sp2 */ ... printf("%s\n", sp2); /* We can still derefrence sp2 */ ... shFree(sp2) /* Now we will actually de-allocate the block. Any attempts to dereference it from here on out will result in failure */

    shIsShMallocPtr

    shIsShMallocPtr() determines if a given address has been allocated using shMalloc() or not. SYNOPSIS: #include "shCGarbage.h" int shIsShMallocPtr(void *ptr) ptr - address of interest RETURNS: On success - 1 On failure - 0

    shMemSerialCB

    shMemSerialCB() registers a user-supplied callback function which is invoked on a specified event. The event of interest here is the allocation of a memory block with a certain serial number. The signature of the user-supplied callback function should be as follows: void callBackFunc(unsigned long thresh, const SH_MEMORY *m) Unless the callBackFunc() calls exit(2) or abort(2), control will be passed back to the Dervish memory management routines and execution will continue from the point where the callback function was called.

    The arguments to the callback function are the threshold, and the block of memory that's just been allocated.

    If the requested serial number is ~0UL (all bits set), the callback function will be called during every shMalloc call.

    SYNOPSIS: #include "shCGarbage.h" void shMemSerialCB(const unsigned long trigger, void (*callBackFuncp)(unsigned long thresh, const SH_MEMORY *m)) trigger - serial number to trigger the event on callBackFuncp - address of the user-supplied callback function RETURNS: Nothing. EXAMPLE: /* * Suppose you want to set a certain global flag when memory block number * 1000 is being allocated. Here's how to do it... */ void callBackFunc(void); static int flag = 0; void callBackFunc(void) { flag = 1; } int main(int ac, char *av[]) { ... shMemSerialCB(1000L, callBackFunc); ... /* * Do regular processing that may call shMalloc() many times */ ... if (flag) ... else ... } For an example of a more sophisticated callback, see the discussion of p_shMemCheck.

    Rather than call shMemSerialCB to change the threshold, it is often convenient to set the global variable g_Serial_threshold using a debugger.


    shMemSerialFreeCB

    shMemSerialFreeCB() registers a user-supplied callback function which is called when a memory block with a certain serial number is freed. The signature of the user-supplied callback function should be as follows: void callBackFunc(unsigned long thresh, const SH_MEMORY *m) Unless the callBackFunc() calls exit(2) or abort(2), control will be passed back to the Dervish memory management routines and execution will continue from the point where the callback function was called.

    The arguments to the callback function are the threshold, and the block of memory that's just about to be freed.

    SYNOPSIS: #include "shCGarbage.h" void shMemSerialFreeCB(const unsigned long trigger, void (*callBackFuncp)(unsigned long thresh, const SH_MEMORY *m)) trigger - serial number to trigger the event on callBackFuncp - address of the user-supplied callback function RETURNS: Nothing. For an example of using callbacks, see shMemSerialCB.

    Rather than call shMemSerialFreeCB to change the threshold, it is often convenient to set the global variable g_Serial_free_threshold using a debugger.


    shMemFreeBlocks

    shMemFreeBlocks() performs the unenviable task of garbage collection in Dervish Memory management routines. Unlike other operating environments (LISP, SmallTalk, emacs), where garbage collection is an automatic procedure, Dervish garbage collection is user driven.

    Central to the theme of garbage collection in Dervish is the idea that each memory block allocated using shMalloc() has a unique serial number associated with it. Garbage collection then simply becomes an issue of deallocating all memory blocks bounded (inclusively) by two serial numbers.

    Note that you must be absolutely sure that all blocks in this range are actually not in use before calling this function! In practice this probably requires you to free them individually rather than using shMemFreeBlocks. You can check this within TCL with a command such as

       if {[memBlocksGetRange $low $high] != {}} { error "Not all memory is free" }
    

    To deallocate a single memory block, set hi_bound to be the same as low_bound

    SYNOPSIS: #include "shCGarbage.h" int shMemFreeBlocks(const unsigned long low_bound, const unsigned long hi_bound, void (*funcp)(void *)) low_bound - deallocate blocks starting with this serial number hi_bound - stop deallocating blocks whose serial number is greater then hi_bound funcp - pointer to a user-supplied function to deallocate memory blocks. Can be NULL, in which case, shFree() is used. RETURNS: On success - 0 On failure - 1 : if the AMP (allocated memory pool) is empty 2 : if low_bound >= hi_bound

    shMemStatsPrint

    shMemStatsPrint() prints (to stdandard output) some interesting memory usage statistics. SYNOPSIS: #include "shCGarbage.h" void shMemStatsPrint(void) RETURNS: Nothing

    shMemSerialNumberGet

    shMemSerialNumberGet() returns the serial number of the memory block last allocated. SYNOPSIS: #include "shCGarbage.h" unsigned long shMemSerialNumberGet(void)

    shMemTotalBytesMalloced

    shMemTotalBytesMalloced() returns the total number of bytes allocated from the operating system. SYNOPSIS: #include "shCGarbage.h" unsigned long shMemTotalBytesMalloced(void)

    shMemActualBytesMalloced

    shMemActualBytesMalloced() returns the total number of bytes provided to the AMP and FMP. SYNOPSIS: #include "shCGarbage.h" unsigned long shMemActualBytesMalloced(void)

    shMemBytesInUse

    shMemBytesInUse() returns the total number of bytes in the allocated memory pool. SYNOPSIS: #include "shCGarbage.h" unsigned long shMemBytesInUse(void)

    shMemBytesInPool

    shMemBytesInUse() returns the total number of bytes in the free memory pool. Note that return result of this function, if added to the return result of shMemBytesInUse() gives a count of the total number of bytes being managed by Dervish. SYNOPSIS: #include "shCGarbage.h" unsigned long shMemBytesInPool(void)

    shMemNumMallocs

    shMemNumMallocs() returns the total number of times shMalloc() was called. SYNOPSIS: #include "shCGarbage.h" unsigned long shMemNumMallocs(void)

    shMemNumFrees

    shMemNumFrees() returns the total number of times shFree() was called. SYNOPSIS: #include "shCGarbage.h" unsigned long shMemNumFrees(void)

    shMemAMPSize

    shMemAMPSize() returns the total number of memory blocks on the Allocated Memory Pool. SYNOPSIS: #include "shCGarbage.h" unsigned long shMemAMPSize(void)

    p_shMemCheck

    shMemCheck() checks the memory pool for corruption. SYNOPSIS: #include "dervish.h" int p_shMemCheck(int check_allocated, /* check allocated blocks? */ int check_free, /* check free blocks? */ int abort_on_error); /* abort on first error? */ Calling this routine checks all the memory known to dervish; this may take some time. Note that there is NO GUARANTEE that a trashed heap will not lead to SEGVs in this code --- but of course this will only happen if you have a bug anyway.

    The number of bad blocks detected is returned

    It's possible to call this from within a memory callback function to achieve any desired granularity of memory checking, e.g.

    static void malloc_check(unsigned long thresh, const SH_MEMORY *mem) { static int abort_on_error = 1; static int check_allocated = 1; static int check_free = 1; static int frequency = 10; shAssert(mem != NULL); if(frequency > 0) { p_shMemCheck(check_allocated, check_free, abort_on_error); shMemSerialCB(thresh + frequency, malloc_check); } } After which, calling shMemSerialCB(1, malloc_check); will cause the heap to be checked every 10 calls to shMalloc

    shMemBlocksizeSet

    Set the minimum size in bytes for a malloc() request, the balance of any memory requested goes onto the internal free list g_mallocList

    Returns the old value; as a special case is size of ~0UL, the value is not changed.

    The initial value of the memory block size is 0; the significance of the call is explained under shMemDefragment.

    SYNOPSIS: #include "dervish.h" size_t shMemBlocksizeSet(size_t size) /* size of requests from to malloc */

    shMemDefragment

    shMemCheck() checks the memory pool for corruption. Defragment the free memory list.

    You have two options; if free_to_os is true, simply go through the free lists freeing all allocated blocks back to the O/S, so as to give it a chance to defragment for us. To use this option you must not have called shMemBlocksizeSet or chaos will ensue, as some of the freed blocks will not have been allocated directly by malloc.

    Otherwise, go through the FMP looking for adjacent blocks which are then merged together, and if possible returned to g_mallocList. This will only work if large blocks have been allocated from the O/S; i.e. if you called shMemBlocksizeSet with a largish argument (a few Mby?).

    Return 0 if all is well, or -1 in case of trouble

    SYNOPSIS: #include "dervish.h" int shMemDefragment(int free_to_os) /* return free memory to O/S? */