Preserving Message Ordering
Module private
locks cannot be used to preserve message ordering because they cannot be held
across calls to putnext(9F)
and the other messages that pass routines to other modules. The alternatives
for preserving message ordering are:
Use MT STREAMS perimeters.
Pass all messages through the service procedures.
The service procedure can drop the locks before calling putnext(9F)
or qreply(9F),
without reordering messages, because the framework guarantees that at most,
one thread will execute in the service procedure for a
given queue.
Use perimeters to avoid the performance penalty for using service procedures.
Preparing to Port
When modifying a STREAMS
driver to take advantage of the multithreaded kernel, a level of MT safety
is selected according to:
The desired degree of concurrency
The natural concurrency of the underlying module
The amount of effort or complexity required
Much of the effort in conversion is simply determining the appropriate
degree of data sharing and the corresponding granularity of locking (see Table 12-1). The actual time spent configuring perimeters
and/or installing locks should be much smaller than the time spent in analysis.
To port your module, you must understand the data structures used within
your module, as well as the accesses to those data structures. You must fully
understand the relationship between all portions of the module and private
data within that module, and to use the MT STREAMS perimeters (or the synchronization
primitives available) to maintain the integrity of these private data structures.
You must explicitly restrict access to private module data structures
as appropriate to ensure the integrity of these data structures. You must
use the MT STREAMS perimeters to restrict the concurrency in the module so
that the parts of the module that modify private data are single-threaded
with respect to the parts of the module that read the same data. (For more
information about perimeters, see MT STREAMS Perimeters.) Besides
perimeters, you can use the synchronization primitives available (mutex, condition variables, readers/writer, semaphore) to explicitly
restrict access to module private data appropriate for the operations within
the module on that data.
The first step in multithreading a module or driver is to analyze the
module, breaking the entire module up into a list of individual operations
and the private data structures referenced in each operation. Part of this
first step is deciding upon a level of concurrency for the module. Ask yourself
which of these operations can be multithreaded and which must be single-threaded.
Try to find a level of concurrency that is "natural" for the module
and matches one of the available perimeters (or, alternatively, requires the
minimal number of locks) , and has a simple and straightforward implementation.
Avoid additional unnecessary complexity.
Typical questions to ask are:
What data structures are maintained within the module?
What types of accesses are made to each field of these data
structures?
When is each data structure accessed destructively (written)
and when is it accessed non-destructively (read)?
Which operations within the module should be allowed to execute
concurrently?
Is per module single-threading appropriate for the module?
Is per queue-pair or per queue single-threading appropriate?
What are the message ordering requirements?
Porting to the SunOS 5 System
When porting a STREAMS module or driver from the SunOS 4 system to
the SunOS 5 system, the module should be examined with respect to the following
areas:
For portability and correct operation, each module must adhere to the
SunOS DDI/DKI. Several facilities available in previous releases of the SunOS
system have changed and can take different arguments, or produce different
side effects, or no longer exist in the SunOS 5 system. The module writer
should carefully review the module with respect to the DDI/DKI.
Each module that accesses underlying Sun-specific features included
in the SunOS 5 system should conform to the Device Driver Interface. The SunOS
5 DDI defines the interface used by the device driver to register device hardware
interrupts, access device node properties, map device slave memory, and establish
and synchronize memory mappings for DVMA (Direct Virtual Memory Access). These
areas are primarily applicable to hardware device drivers. Refer to the Device
Driver Interface Specification within the Writing Device Drivers for details on the SunOS 5 DDI
and DVMA.
The kernel networking subsystem in the SunOS 5 system is based on STREAMS.
Datalink drivers that used the ifnet interface in the SunOS
4 system must be converted to use DLPI for the SunOS 5 system. Refer to the
Data Link Provider Interface, Revision 2 specification.
After reviewing the module for conformance to the SunOS
5 DKI and DDI specifications, you should be able to consider the impact of
multithreading on the module.
Sample Multithreaded Device Driver Using a Per Module Inner Perimeter
Example 12-1 is a sample multithreaded, loadable,
STREAMS pseudo-driver. The driver MT design is the simplest possible based
on using a per module inner perimeter. Thus, only one thread can execute in
the driver at any time. In addition, a quntimeout(9F) synchronous callback routine is used.
The driver cancels an outstanding qtimeout(9F) by calling quntimeout(9F) in the close routine. See close() Race Conditions.
Example 12-1 Multithreaded, Loadable, STREAMS Pseudo-Driver
/*
* Example SunOS 5 multithreaded STREAMS pseudo device driver.
* Using a D_MTPERMOD inner perimeter.
*/
#include <sys/types.h>
#include <sys/errno.h>
#include <sys/stropts.h>
#include <sys/stream.h>
#include <sys/strlog.h>
#include <sys/cmn_err.h>
#include <sys/modctl.h>
#include <sys/kmem.h>
#include <sys/conf.h>
#include <sys/ksynch.h>
#include <sys/stat.h>
#include <sys/ddi.h>
#include <sys/sunddi.h>
/*
* Function prototypes.
*/
static int xxidentify(dev_info_t *);
static int xxattach(dev_info_t *, ddi_attach_cmd_t);
static int xxdetach(dev_info_t *, ddi_detach_cmd_t);
static int xxgetinfo(dev_info_t *,ddi_info_cmd_t,void *,void**);
static int xxopen(queue_t *, dev_t *, int, int, cred_t *);
static int xxclose(queue_t *, int, cred_t *);
static int xxwput(queue_t *, mblk_t *);
static int xxwsrv(queue_t *);
static void xxtick(caddr_t);
/*
* Streams Declarations
*/
static struct module_info xxm_info = {
99, /* mi_idnum */
"xx", /* mi_idname */
0, /* mi_minpsz */
INFPSZ, /* mi_maxpsz */
0, /* mi_hiwat */
0 /* mi_lowat */
};
static struct qinit xxrinit = {
NULL, /* qi_putp */
NULL, /* qi_srvp */
xxopen, /* qi_qopen */
xxclose, /* qi_qclose */
NULL, /* qi_qadmin */
&xxm_info, /* qi_minfo */
NULL /* qi_mstat */
};
static struct qinit xxwinit = {
xxwput, /* qi_putp */
xxwsrv, /* qi_srvp */
NULL, /* qi_qopen */
NULL, /* qi_qclose */
NULL, /* qi_qadmin */
&xxm_info, /* qi_minfo */
NULL /* qi_mstat */
};
static struct streamtab xxstrtab = {
&xxrinit, /* st_rdinit */
&xxwinit, /* st_wrinit */
NULL, /* st_muxrinit */
NULL /* st_muxwrinit */
};
/*
* define the xx_ops structure.
*/
static struct cb_ops cb_xx_ops = {
nodev, /* cb_open */
nodev, /* cb_close */
nodev, /* cb_strategy */
nodev, /* cb_print */
nodev, /* cb_dump */
nodev, /* cb_read */
nodev, /* cb_write */
nodev, /* cb_ioctl */
nodev, /* cb_devmap */
nodev, /* cb_mmap */
nodev, /* cb_segmap */
nochpoll, /* cb_chpoll */
ddi_prop_op, /* cb_prop_op */
&xxstrtab, /* cb_stream */
(D_NEW|D_MP|D_MTPERMOD) /* cb_flag */
};
static struct dev_ops xx_ops = {
DEVO_REV, /* devo_rev */
0, /* devo_refcnt */
xxgetinfo, /* devo_getinfo */
xxidentify, /* devo_identify */
nodev, /* devo_probe */
xxattach, /* devo_attach */
xxdetach, /* devo_detach */
nodev, /* devo_reset */
&cb_xx_ops, /* devo_cb_ops */
(struct bus_ops *)NULL /* devo_bus_ops */
};
/*
* Module linkage information for the kernel.
*/
static struct modldrv modldrv = {
&mod_driverops, /* Type of module. This one is a driver */
"xx", /* Driver name */
&xx_ops, /* driver ops */
};
static struct modlinkage modlinkage = {
MODREV_1,
&modldrv,
NULL
};
/*
* Driver private data structure. One is allocated per Stream.
*/
struct xxstr {
struct xxstr *xx_next; /* pointer to next in list */
queue_t *xx_rq; /* read side queue pointer */
minor_t xx_minor; /* minor device # (for clone) */
int xx_timeoutid; /* id returned from timeout() */
};
/*
* Linked list of opened Stream xxstr structures.
* No need for locks protecting it since the whole module is
* single threaded using the D_MTPERMOD perimeter.
*/
static struct xxstr *xxup = NULL;
/*
* Module Config entry points
*/
_init(void)
{
return (mod_install(&modlinkage));
}
_fini(void)
{
return (mod_remove(&modlinkage));
}
_info(struct modinfo *modinfop)
{
return (mod_info(&modlinkage, modinfop));
}
/*
* Auto Configuration entry points
*/
/* Identify device. */
static int
xxidentify(dev_info_t *dip)
{
if (strcmp(ddi_get_name(dip), "xx") == 0)
return (DDI_IDENTIFIED);
else
return (DDI_NOT_IDENTIFIED);
}
/* Attach device. */
static int
xxattach(dev_info_t *dip, ddi_attach_cmd_t cmd)
{
/* This creates the device node. */
if (ddi_create_minor_node(dip, "xx", S_IFCHR, ddi_get_instance(dip),
DDI_PSEUDO, CLONE_DEV) == DDI_FAILURE) {
return (DDI_FAILURE);
}
ddi_report_dev(dip);
return (DDI_SUCCESS);
}
/* Detach device. */
static int
xxdetach(dev_info_t *dip, ddi_detach_cmd_t cmd)
{
ddi_remove_minor_node(dip, NULL);
return (DDI_SUCCESS);
}
/* ARGSUSED */
static int
xxgetinfo(dev_info_t *dip, ddi_info_cmd_t infocmd, void *arg,
void **resultp)
{
dev_t dev = (dev_t) arg;
int instance, ret = DDI_FAILURE;
devstate_t *sp;
state *statep;
instance = getminor(dev);
switch (infocmd) {
case DDI_INFO_DEVT2DEVINFO:
if ((sp = ddi_get_soft_state(statep,
getminor((dev_t) arg))) != NULL) {
*resultp = sp->devi;
ret = DDI_SUCCESS;
} else
*result = NULL;
break;
case DDI_INFO_DEVT2INSTANCE:
*resultp = (void *)instance;
ret = DDI_SUCCESS;
break;
default:
break;
}
return (ret);
}
static
xxopen(rq, devp, flag, sflag, credp)
queue_t *rq;
dev_t *devp;
int flag;
int sflag;
cred_t *credp;
{
struct xxstr *xxp;
struct xxstr **prevxxp;
minor_t minordev;
/* If this stream already open - we're done. */
if (rq->q_ptr)
return (0);
/* Determine minor device number. */
prevxxp = & xxup;
if (sflag == CLONEOPEN) {
minordev = 0;
while ((xxp = *prevxxp) != NULL) {
if (minordev < xxp->xx_minor)
break;
minordev++;
prevxxp = &xxp->xx_next;
}
*devp = makedevice(getmajor(*devp), minordev)
} else
minordev = getminor(*devp);
/* Allocate our private per-Stream data structure. */
if ((xxp = kmem_alloc(sizeof (struct xxstr), KM_SLEEP)) == NULL)
return (ENOMEM);
/* Point q_ptr at it. */
rq->q_ptr = WR(rq)->q_ptr = (char *) xxp;
/* Initialize it. */
xxp->xx_minor = minordev;
xxp->xx_timeoutid = 0;
xxp->xx_rq = rq;
/* Link new entry into the list of active entries. */
xxp->xx_next = *prevxxp;
*prevxxp = xxp;
/* Enable xxput() and xxsrv() procedures on this queue. */
qprocson(rq);
return (0);
}
static
xxclose(rq, flag, credp)
queue_t *rq;
int flag;
cred_t *credp;
{
struct xxstr *xxp;
struct xxstr **prevxxp;
/* Disable xxput() and xxsrv() procedures on this queue. */
qprocsoff(rq);
/* Cancel any pending timeout. */
xxp = (struct xxstr *) rq->q_ptr;
if (xxp->xx_timeoutid != 0) {
(void) quntimeout(rq, xxp->xx_timeoutid);
xxp->xx_timeoutid = 0;
}
/* Unlink per-stream entry from the active list and free it. */
for (prevxxp = &xxup; (xxp = *prevxxp) != NULL;
prevxxp = &xxp->xx_next)
if (xxp == (struct xxstr *) rq->q_ptr)
break;
*prevxxp = xxp->xx_next;
kmem_free (xxp, sizeof (struct xxstr));
rq->q_ptr = WR(rq)->q_ptr = NULL;
return (0);
}
static
xxwput(wq, mp)
queue_t *wq;
mblk_t *mp;
{
struct xxstr *xxp = (struct xxstr *)wq->q_ptr;
/* write your code here */
/* *** Sacha's Comments *** broken */
freemsg(mp);
mp = NULL;
if (mp != NULL)
putnext(wq, mp);
}
static
xxwsrv(wq)
queue_t *wq;
{
mblk_t *mp;
struct xxstr *xxp;
xxp = (struct xxstr *) wq->q_ptr;
while (mp = getq(wq)) {
/* write your code here */
freemsg(mp);
/* for example, start a timeout */
if (xxp->xx_timeoutid != 0) {
/* cancel running timeout */
(void) quntimeout(wq, xxp->xx_timeoutid);
}
xxp->xx_timeoutid = qtimeout(wq, xxtick, (char *)xxp, 10);
}
}
static void
xxtick(arg)
caddr_t arg;
{
struct xxstr *xxp = (struct xxstr *)arg;
xxp->xx_timeoutid = 0; /* timeout has run */
/* write your code here */
}
|
|
Previous