programming

Starting a Freecycle module

Blog

I started my Freecycle module for drupal. You can see an example of it working on my site, though it's in a state of flux and may not be working at any given point.

Freecycle is a growing, grassroots movement that reduces landfill trash by promoting the free exchange of used yet still useable goods. In other words, "One man's trash is another man's treasure."

The basic concept is that some used goods can be used by other people. Rather than throwing out usable goods, an owner can post the item to a list, offering it to others. If another person has need of the goods, s/he can respond requesting the used goods.

Part of the problem I have with the process is the difficulty with selecting one person to give the item to, or asking for an item (oooo! pick me! I want it! I need it. I hate sob-story emails from strangers.). When I post items (and I've posted a lot to my local group), I often get a flood of emails. I then have to figure out which person I should give the item to, arrange for pickup, wait to see if they pickup (no shows are a big deal), and reoffer if the items aren't picked up. I think the "Sorry, already taken." emails after the first n emails are received (where n varies on how much I think someone really wants the items and will be likely to pick them up) suck the most.

This Freecycle module will alleviate some of those issues, by having people sign up online. I'll be able to configure how many emails I accept before automatically terminating the list, provide a giveaway/pickup status for unclaimed items, limit how many items someone can pick up (by email address, IP address, etc.) and provide feedback (ala ebay) about no-shows.

Nothing like scratching an itch for the common good.

Project chartering notes

Book page
From a person who managed to attend the Bay Area XP December session about Project Charting. I note that Brian Slesinsky uses the same email address schema I do (with both .org and -yahoo).

More information on yahoogroups.com/groups/bayxp

A project charter is an agreement between the developers and 
the gold-owner (project funder, money spender).

- No technology, protocol, user interface, or other design 
  decisions.  
 
  The charter assumes the project will build a black box 
  containing perfect technology.

Prerequisites for charter:

Vision: hazy statement of the overall company goal (1 sentence)
Mission: the direction we will take to achieve that goal (1 sentence)

The Vision and Mission are persistent across multiple projects.

Project Charter components:

External Objectives:
   - not a feature list or a list of stories
   - a binary, measurable way of evaluating the product or 
     service (success or failure)
   - has an assessment date attached (time at which we 
     measure)
   - may be multiple, sequential objectives (milestones), or 
     repeated assessments
   - out of the team or gold-owner's direct control
   - hard evidence used by gold-owner to justify expense
   Examples:
     - car gets X miles per gallon (but be careful not to 
       make it a feature list)
     - survey of beta-testers shows that 90% are satisfied 
       with the beta version
     - three out of the top five consumer magazines give it
       a positive review
     - three of five main suppliers order the product
       (but typically not sales goals since that's too 
       late/indirect for software developers)

  Internal Objectives:
   - things like improving reuse, process maturity, etc.
   
  Project Boundaries:
   - names the external actors
   - names their inputs and outputs
     (doesn't include technology; no specified protocol)
   - events:
   - actor-initiated, detectable events
   - scheduled events (e.g. due date arrives)

Committed Resources:
   - money, people's time, tools
   - work environment
   - access to information
   - access to decision-makers
   - permission to iterate (don't ship the demo)
   - agreement to re-negotiate charter if a committed 
     resource becomes unavailable
   - the developers must say no if committed resources are
     insufficient

Authorizing players:
   - must be able to make a decision on two questions, and 
     make it stick:
        - Is what we've done so far okay?
        - Can we continue?
        - approve and champion objectives
        - must be actual people, not policy or job titles

MySQL LEFT JOIN: selecting records with no corresponding entries in second table

Book page

Syntax:

SELECT u.uid, u.name, m.id, p.value FROM mpulreg m, users u LEFT JOIN profile_values p ON p.uid = u.uid WHERE u.uid = m.regnum AND p.uid = u.uid AND p.fid='3' AND m.status != 'D' ORDER BY u.name

This says, select the fields from the tables and select profile_values.value where profile_values.uid = users.uid, even if it's empty. So, you'll get an empty value for the profile_values.value if it doesn't exist.

Detailed explanation

You have two or more tables and would like to get entries from all the tables in one SELECT statement. However, one of the tables doesn't have any entries that correspond to another table with entries in it.

For example, let's say you have a username table, an event registration table and an user real name table. The users table contains a few users; the event contains information about who has signed up for an event; and the real name table shows the username's real names:

+------------------+  +----------------+  +---------------------+
| users            |  | events         |  | names               |
+-----+------------+  +----+-----+-----+  +----+-----+----------+
| uid | name       |  | id | eid | uid |  | id | uid | realname |    
+-----+------------+  +----+-----+-----+  +----+-----+----------+
| 1   | kristofer  |  | 1  | 1   | 3   |  | 1  | 1   | Kris McQ |
+-----+------------+  +----+-----+-----+  +----+-----+----------+
| 2   | kitt       |  | 2  | 1   | 1   |  | 2  | 2   | Kate McQ |
+-----+------------+  +----+-----+-----+  +----+-----+----------+
| 3   | elizabeth  |  | 3  | 1   | 2   |  
+-----+------------+  +----+-----+-----+  

In this case, 'kitt' is going to event 1 (with eid = 1), as are 'kristofer' and 'elizabeth'. Similarly, kristofer's real name is 'Kris McQ', and kitt's real name is 'Kate McQ'. We don't know elizabeth's real name.

Now, we want to list all the people and see if they are coming to the event, what their real names are:

To do so, first SELECT on the users and events, and we see that everyone is going.

SELECT u.*, e.* FROM users u, events e WHERE u.id = e.uid;
...

If we just put in the real names table, we'll get the wrong information:

SELECT u.*, e.*, r.* FROM users u, events e, realnames r WHERE u.id = e.uid;
...

Instead we want to use a LEFT JOIN to join two tables, then use that result set on the selection of data from the third table:

SELECT u.*, e.*, r.* FROM users u, events e LEFT JOIN realnames r ON r.uid = e.uid WHERE u.id = e.uid;

Basic Postgresql Commands

Book page

Command

Result

\d

 

\dt

List all tables

\di

List all indexes

\ds

List all sequences

\dv

List all views

\dS

List all PostgreSQL-defined tables

\d table-name

Show table definition

\d index-name

Show index definition

\d view-name

Show view definition

\d sequence-name

Show sequence definition

\dp

List all privileges

\dl

List all large objects

\da

List all aggregates

\df

List all functions

\df function-name

List all functions with given name

\do

List all operators

\do operator-name

List all operators with given name

\dT

List all types

\l

List all databases in this cluster

MySQL Tricks and Tips

Book page

MySQL tricks and tips.

Also known as information I used frequently and have to look up each time, so why not put it all in one place?

Most of the information on these pages work for any SQL database. Comment structures are different among databases, as are administrative commands ('show tables' in MySQL vs '\d' in PostgreSQL). The rest, however, is the same.

Counting the Number of Bits in a Byte

Book page

From: http://www-db.stanford.edu/~manku/bitcount/bitcount.html

Counting Number of On Bits in an Integer

Fast Bit Counting Routines

Compiled from various sources by Gurmeet Singh Manku

A common problem asked in job interviews is to count the number of bits that are on in an unsigned integer. Here are seven solutions to this problem. Source code in C is available.

Iterated Count

   int bitcount (unsigned int n)  
   {
      int count=0;    
      while (n)
      {
         count += n & 0x1u ;
         n >>= 1 ;
      }
      return count ;
   }
Sparse Ones

   int bitcount (unsigned int n)  
   {  
      int count=0 ;
      while (n)
      {
         count++ ;
         n &= (n - 1) ;
      }
      return count ;
   }
Dense Ones

   int bitcount (unsigned int n)     
   {
      int count = 8 * sizeof(int) ;
      n ^= (unsigned int) -1 ;
      while (n)
      {
         count-- ;
         n &= (n - 1) ;
      }
      return count ;
   }
Precompute_8bit

   // static int bits_in_char [256] ;           
   
   int bitcount (unsigned int n)
   {
      // works only for 32-bit ints
    
      return bits_in_char [n         & 0xffu]
          +  bits_in_char [(n >>  8) & 0xffu]
          +  bits_in_char [(n >> 16) & 0xffu]
          +  bits_in_char [(n >> 24) & 0xffu] ;
   }
Iterated Count runs in time proportional to the total number of bits. It simply loops through all the bits, terminating slightly earlier because of the while condition. Useful if 1's are sparse and among the least significant bits. Sparse Ones runs in time proportional to the number of 1 bits. The line n &= (n - 1) simply sets the rightmost 1 bit in n to 0. Dense Ones runs in time proportional to the number of 0 bits. It is the same as Sparse Ones, except that it first toggles all bits (n ~= -1), and continually subtracts the number of 1 bits from sizeof(int). Precompute_8bit assumes an array bits_in_char such that bits_in_char[i] contains the number of 1 bits in the binary representation for i. It repeatedly updates count by masking out the last eight bits in n, and indexing into bits_in_char.

Precompute_16bit

  // static char bits_in_16bits [0x1u > 16) & 0xffffu] ;
  }

Precompute_16bit is a variant of Precompute_8bit in that an array bits_in_16bits[] stores the number of 1 bits in successive 16 bit numbers (shorts).

Parallel Count

  #define TWO(c)     (0x1u > (TWO(c))) & MASK(c))
     
  int bitcount (unsigned int n)
  {
     n = COUNT(n, 0) ;
     n = COUNT(n, 1) ;
     n = COUNT(n, 2) ;
     n = COUNT(n, 3) ;
     n = COUNT(n, 4) ;
     /* n = COUNT(n, 5) ;    for 64-bit integers */
     return n ;
  }
Parallel Count carries out bit counting in a parallel fashion. Consider n after the first line has finished executing. Imagine splitting n into pairs of bits. Each pair contains the number of ones in those two bit positions in the original n. After the second line has finished executing, each nibble contains the number of ones in those four bits positions in the original n. Continuing this for five iterations, the 64 bits contain the number of ones among these sixty-four bit positions in the original n. That is what we wanted to compute.

Nifty Parallel Count

 #define MASK_01010101 (((unsigned int)(-1))/3)
 #define MASK_00110011 (((unsigned int)(-1))/5)
 #define MASK_00001111 (((unsigned int)(-1))/17)

 int bitcount (unsigned int n)
 {
    n = (n & MASK_01010101) + ((n >> 1) & MASK_01010101) ; 
    n = (n & MASK_00110011) + ((n >> 2) & MASK_00110011) ; 
    n = (n & MASK_00001111) + ((n >> 4) & MASK_00001111) ; 
    return n % 255 ;
 }
Nifty Parallel Count works the same way as Parallel Count for the first three iterations. At the end of the third line (just before the return), each byte of n contains the number of ones in those eight bit positions in the original n. A little thought then explains why the remainder modulo 255 works.

MIT HACKMEM Count

 int bitcount(unsigned int n)                          
 {
    /* works for 32-bit numbers only    */
    /* fix last line for 64-bit numbers */
    
    register unsigned int tmp;
    
    tmp = n - ((n >> 1) & 033333333333)
            - ((n >> 2) & 011111111111);
    return ((tmp + (tmp >> 3)) & 030707070707) % 63;
 }
MIT Hackmem Count is funky. Consider a 3 bit number as being 4a+2b+c. If we shift it right 1 bit, we have 2a+b. Subtracting this from the original gives 2a+b+c. If we shift the original 2 bits right we get a, and so with another subtraction we have a+b+c, which is the number of bits in the original number. How is this insight employed? The first assignment statement in the routine computes tmp. Consider the octal representation of tmp. Each digit in the representation is simply the number of 1's in the corresponding three bit positions in n. The last return statement sums these octal digits to produce the final answer. The key idea is to add adjacent pairs of octal digits together and then compute the remainder modulus 63. This is accomplished by right-shifting tmp by three bits, adding it to tmp itself and ANDing with a suitable mask. This yields a number in which groups of six adjacent bits (starting from the LSB) contain the number of 1's among those six positions in n. This number modulo 63 yields the final answer. For 64-bit numbers, we would have to add triples of octal digits and use modulus 1023. This is HACKMEM 169, as used in X11 sources. Source: MIT AI Lab memo, late 1970's.

     No Optimization         Some Optimization       Heavy Optimization

  Precomp_16 52.94 Mcps    Precomp_16 76.22 Mcps    Precomp_16 80.58 Mcps  
   Precomp_8 29.74 Mcps     Precomp_8 49.83 Mcps     Precomp_8 51.65 Mcps
    Parallel 19.30 Mcps      Parallel 36.00 Mcps      Parallel 38.55 Mcps
         MIT 16.93 Mcps           MIT 17.10 Mcps         Nifty 31.82 Mcps
       Nifty 12.78 Mcps         Nifty 16.07 Mcps           MIT 29.71 Mcps
      Sparse  5.70 Mcps        Sparse 15.01 Mcps        Sparse 14.62 Mcps
       Dense  5.30 Mcps        Dense  14.11 Mcps         Dense 14.56 Mcps
    Iterated  3.60 Mcps      Iterated  3.84 Mcps      Iterated  9.24 Mcps

  Mcps = Million counts per second
Which of the several bit counting routines is the fastest? Results of speed trials on an i686 are summarized in the table on left. "No Optimization" was compiled with plain gcc. "Some Optimizations" was gcc -O3. "Heavy Optimizations" corresponds to gcc -O3 -mcpu=i686 -march=i686 -fforce-addr -funroll-loops -frerun-cse-after-loop -frerun-loop-opt -malign-functions=4.
Gurmeet Singh Manku
Last update: 27 Jul 2002

Pages