Really depends on what you're into. Amsterdam is a mecca for pot heads, an interesting place to visit for others.
If you're the latter, 2 day is MORE than enough. Day 1 you can wander the downtown, 'enjoy' the cafe's etc. and then party at night. Day 2, enjoy the cafe's some more, then hit...