Dervish Memory Management and Garbage Collection C API
Dervish provides an extensive API for memory management. The following
discussions details each API and wherever appropriate, source code examples
are included as well.
For an overview of DERVISH Memory Management and Garbage Collection, and the
definition of some commonly used terms,
press here.
Descriptions of the C API
shMalloc
shFree
shCalloc
shRealloc
shMemEmptyCB
shMemRefCntrIncr
shMemRefCntrDecr
shIsShMallocPtr
shMemSerialCB
shMemSerialFreeCB
shMemFreeBlocks
shMemStatsPrint
shMemSerialNumberGet
shMemTotalBytesMalloced
shMemBytesInUse
shMemBytesInPool
shMemNumFrees
shMemNumMallocs
shMemAMPSize
p_shMemCheck
shMemBlocksizeSet
shMemDefragment
Allocate requested bytes of memory, returning address of the starting byte
to the user. The returned pointer is guaranteed to be suitably aligned for
any type of object.
SYNOPSIS:
#include "shCGarbage.h"
void *shMalloc(const size_t size)
size - number of bytes to allocate
RETURNS:
On success - pointer to a suitably aligned memory area
On failure - there is no failure return from this function. It will abort
if it cannot allocate requested memory
De-allocate memory block specified in the parameter. This function will abort
if:
address of the block to be de-allocated has not been allocated by
shMalloc(), or
attempt is made to shFree() a block that has already been
deallocated.
If a memory block to be de-allocated is still being referenced, it
will not be de-allocated. Rather, the reference counter associated with that
block will be decremented. A memory block will only be de-allocated when
it's reference count reaches 0, i.e. it is not being referenced at all.
SYNOPSIS:
#include "shCGarbage.h"
void shFree(void *ptr)
ptr - Pointer to the memory block to be de-allocated
RETURNS:
Nothing
Allocates space for an array with the number of elements specified by the
num_of_elts parameter, where each element is of the size specified by the
elt_size parameter. The space is initialized to all-zero-bits, which
on most but not all machines is equivalent to assignment to a zero of the
appropriate type.
SYNOPSIS:
#include "shCGarbage.h"
void *shCalloc(size_t size)
void *shCalloc(size_t num_of_elts,size_t elt_size);
num_of_elts - number of elements to allocate
elt_size - size of each element
RETURNS:
On success - pointer to a suitably aligned memory area
On failure - there is no failure return from this function. It will abort
if it cannot allocate requested memory
shRealloc() takes a pointer to the memory block previously allocated
by shMalloc() and changes its size while preserving its contents;
Although the system realloc may be able to grow memory blocks in place,
shRealloc never does.
If the new size is less then the original size, contents will be preserved upto
the new size only. If the new size is 0 and the original pointer is not NULL,
the original pointer is de-allocated and a NULL is returned. If the original
pointer is NULL, a new memory block of the requested size is created and
returned.
SYNOPSIS:
#include "shCGarbage.h"
void *shRealloc(void *ptr, size_t size)
ptr - Original pointer to the original memory area
size - new size
RETURNS:
On success - A pointer to the new memory area
On failure - NULL
EXAMPLE:
void *pMemBlock;
pMemBlock = (char *) shMalloc(20);
...
/*
* Do something with pMemBlock
*/
...
/*
* Now we realized that we need more then 20 bytes, let's re-alloc
*/
pMemBlock = (char *) shRealloc(pMemBlock, 200);
...
shMemEmptyCB() registers a user-supplied callback function which
is called when a request for memory cannot be fulfilled.
The signature of the user-supplied callback function should be as follows:
void *callBackFunc(size_t nbyte)
Unless the callBackFunc() calls exit(2) or abort(2),
control will be passed back to the Dervish memory management routines and
execution will continue from the point where the callback function was called.
If you return, you must return a pointer to the requested number
of bytes. If you don't, there is very little chance that your application
will continue successfully!
SYNOPSIS:
#include "shCGarbage.h"
void shMemEmptyCB(const unsigned long trigger,
void *(*callBackFuncp)(size_t nbyte))
trigger - serial number to trigger the event on
callBackFuncp - address of the user-supplied callback function
RETURNS:
Nothing.
For an example of using callbacks,
see shMemSerialCB.
shMemRefCntrIncr() increments a reference counter associated with the
given memory address. This function will abort if the memory address passed
as the parameter was not allocated using shMalloc().
SYNOPSIS:
#include "shCGarbage.h"
void shMemRefCntrIncr(void *addr)
addr - Memory address whose reference counter is to be incremented
RETURN:
Nothing
EXAMPLE:
char *sp1, *sp2;
...
sp1 = shMalloc(20);
...
/*
* Now we need to set sp2 to sp1, but we want to make sure that if sp1 is
* deallocated, sp2 still remains a valid pointer. How do we do that? we
* increment sp1's reference counter as follows:
*/
sp2 = sp1;
shMemRefCntrIncr(sp1);
...
shMemRefCntrDecr() is the complement of shMemRefCntrIncr(). It
decrements a reference counter associated with the memory address passed in
as the parameter. The memory block will be de-allocated only when the reference
counter reaches 0.
#include "shCGarbage.h"
void shMemCntrDecr(void *addr)
addr - Memory address whose reference counter is to be decremented
RETURNS:
Nothing
EXAMPLE:
char *sp1, *sp2;
...
sp1 = shMalloc(20);
...
sp2 = sp1;
shMemRefCntrIncr(sp1);
...
shFree(sp1); /* This will not deallocate block pointed to by sp1,
since it is still being referenced by sp2 */
...
printf("%s\n", sp2); /* We can still derefrence sp2 */
...
shFree(sp2) /* Now we will actually de-allocate the block. Any attempts
to dereference it from here on out will result in
failure */
shIsShMallocPtr() determines if a given address has been allocated using
shMalloc() or not.
SYNOPSIS:
#include "shCGarbage.h"
int shIsShMallocPtr(void *ptr)
ptr - address of interest
RETURNS:
On success - 1
On failure - 0
shMemSerialCB() registers a user-supplied callback function which is
invoked on a specified event. The event of interest here is the allocation
of a memory block with a certain serial number. The signature of the
user-supplied callback function should be as follows:
void callBackFunc(unsigned long thresh, const SH_MEMORY *m)
Unless the callBackFunc() calls exit(2) or abort(2),
control will be passed back to the Dervish memory management routines and
execution will continue from the point where the callback function was called.
The arguments to the callback function are the threshold, and the block
of memory that's just been allocated.
If the requested serial number is ~0UL
(all bits set), the
callback function will be called during every shMalloc call.
SYNOPSIS:
#include "shCGarbage.h"
void shMemSerialCB(const unsigned long trigger,
void (*callBackFuncp)(unsigned long thresh, const SH_MEMORY *m))
trigger - serial number to trigger the event on
callBackFuncp - address of the user-supplied callback function
RETURNS:
Nothing.
EXAMPLE:
/*
* Suppose you want to set a certain global flag when memory block number
* 1000 is being allocated. Here's how to do it...
*/
void callBackFunc(void);
static int flag = 0;
void callBackFunc(void)
{
flag = 1;
}
int main(int ac, char *av[])
{
...
shMemSerialCB(1000L, callBackFunc);
...
/*
* Do regular processing that may call shMalloc() many times
*/
...
if (flag)
...
else
...
}
For an example of a more sophisticated callback, see the discussion of
p_shMemCheck.
Rather than call shMemSerialCB
to change the threshold, it
is often convenient to set the global variable g_Serial_threshold
using a debugger.
shMemSerialFreeCB() registers a user-supplied callback function which
is called when a memory block with a certain serial number is freed.
The signature of the
user-supplied callback function should be as follows:
void callBackFunc(unsigned long thresh, const SH_MEMORY *m)
Unless the callBackFunc() calls exit(2) or abort(2),
control will be passed back to the Dervish memory management routines and
execution will continue from the point where the callback function was called.
The arguments to the callback function are the threshold, and the block
of memory that's just about to be freed.
SYNOPSIS:
#include "shCGarbage.h"
void shMemSerialFreeCB(const unsigned long trigger,
void (*callBackFuncp)(unsigned long thresh, const SH_MEMORY *m))
trigger - serial number to trigger the event on
callBackFuncp - address of the user-supplied callback function
RETURNS:
Nothing.
For an example of using callbacks,
see shMemSerialCB.
Rather than call shMemSerialFreeCB
to change the threshold, it
is often convenient to set the global variable
g_Serial_free_threshold
using a debugger.
shMemFreeBlocks() performs the unenviable task of garbage collection
in Dervish Memory management routines. Unlike other operating environments
(LISP, SmallTalk, emacs), where garbage collection is an automatic procedure,
Dervish garbage collection is user driven.
Central to the theme of garbage collection in Dervish is the idea that each
memory block allocated using shMalloc() has a unique serial number
associated with it. Garbage collection then simply becomes an issue of
deallocating all memory blocks bounded (inclusively) by two serial numbers.
Note that you must be absolutely sure that all blocks in
this range are actually not in use before calling this function! In
practice this probably requires you to free them individually rather
than using shMemFreeBlocks
. You can check this within
TCL with a command such as
if {[memBlocksGetRange $low $high] != {}} { error "Not all memory is free" }
To deallocate a single memory block, set hi_bound to be the same as
low_bound
SYNOPSIS:
#include "shCGarbage.h"
int shMemFreeBlocks(const unsigned long low_bound,
const unsigned long hi_bound,
void (*funcp)(void *))
low_bound - deallocate blocks starting with this serial number
hi_bound - stop deallocating blocks whose serial number is greater
then hi_bound
funcp - pointer to a user-supplied function to deallocate memory
blocks. Can be NULL, in which case, shFree() is used.
RETURNS:
On success - 0
On failure - 1 : if the AMP (allocated memory pool) is empty
2 : if low_bound >= hi_bound
shMemStatsPrint() prints (to stdandard output) some interesting
memory usage statistics.
SYNOPSIS:
#include "shCGarbage.h"
void shMemStatsPrint(void)
RETURNS:
Nothing
shMemSerialNumberGet() returns the serial number of the memory
block last allocated.
SYNOPSIS:
#include "shCGarbage.h"
unsigned long shMemSerialNumberGet(void)
shMemTotalBytesMalloced() returns the total number of bytes allocated
from the operating system.
SYNOPSIS:
#include "shCGarbage.h"
unsigned long shMemTotalBytesMalloced(void)
shMemActualBytesMalloced() returns the total number of bytes
provided to the AMP and FMP.
SYNOPSIS:
#include "shCGarbage.h"
unsigned long shMemActualBytesMalloced(void)
shMemBytesInUse() returns the total number of bytes in the allocated
memory pool.
SYNOPSIS:
#include "shCGarbage.h"
unsigned long shMemBytesInUse(void)
shMemBytesInUse() returns the total number of bytes in the free
memory pool. Note that return result of this function, if added to the
return result of shMemBytesInUse() gives a count of the total number
of bytes being managed by Dervish.
SYNOPSIS:
#include "shCGarbage.h"
unsigned long shMemBytesInPool(void)
shMemNumMallocs() returns the total number of times shMalloc()
was called.
SYNOPSIS:
#include "shCGarbage.h"
unsigned long shMemNumMallocs(void)
shMemNumFrees() returns the total number of times shFree()
was called.
SYNOPSIS:
#include "shCGarbage.h"
unsigned long shMemNumFrees(void)
shMemAMPSize() returns the total number of memory blocks on the
Allocated Memory Pool.
SYNOPSIS:
#include "shCGarbage.h"
unsigned long shMemAMPSize(void)
shMemCheck() checks the memory pool for corruption.
SYNOPSIS:
#include "dervish.h"
int
p_shMemCheck(int check_allocated, /* check allocated blocks? */
int check_free, /* check free blocks? */
int abort_on_error); /* abort on first error? */
Calling this routine checks all the memory known to dervish; this may
take some time. Note that there is NO GUARANTEE that a trashed heap
will not lead to SEGVs in this code --- but of course this will only
happen if you have a bug anyway.
The number of bad blocks detected is returned
It's possible to call this from within a memory callback function
to achieve any desired granularity of memory checking, e.g.
static void
malloc_check(unsigned long thresh, const SH_MEMORY *mem)
{
static int abort_on_error = 1;
static int check_allocated = 1;
static int check_free = 1;
static int frequency = 10;
shAssert(mem != NULL);
if(frequency > 0) {
p_shMemCheck(check_allocated, check_free, abort_on_error);
shMemSerialCB(thresh + frequency, malloc_check);
}
}
After which, calling
shMemSerialCB(1, malloc_check);
will cause the heap to be checked every 10 calls to shMalloc
Set the minimum size in bytes for a malloc() request, the balance of any memory
requested goes onto the internal free list g_mallocList
Returns the old value; as a special case is size of ~0UL
,
the value is not changed.
The initial value of the memory block size is 0; the significance of the
call is explained under shMemDefragment.
SYNOPSIS:
#include "dervish.h"
size_t
shMemBlocksizeSet(size_t size) /* size of requests from to malloc */
shMemCheck() checks the memory pool for corruption.
Defragment the free memory list.
You have two options; if free_to_os is true, simply go through the
free lists freeing all allocated blocks back to the O/S, so as to give it
a chance to defragment for us. To use this option you must not
have called shMemBlocksizeSet or chaos will ensue, as some of the
freed blocks will not have been allocated directly by malloc.
Otherwise, go through the FMP looking for adjacent blocks which
are then merged together, and if possible returned to g_mallocList.
This will only work if large blocks have been allocated from the
O/S; i.e. if you called
shMemBlocksizeSet with a largish
argument (a few Mby?).
Return 0 if all is well, or -1 in case of trouble
SYNOPSIS:
#include "dervish.h"
int
shMemDefragment(int free_to_os) /* return free memory to O/S? */