Story points are an arbitrary measure of complexity used to calculate and project team velocity. They are not meant to correspond to a measure of time or difficulty. Estimation is typically done as a team, and talking though upcoming features in order to estimate them is a valuable exercise. It gives everyone familiarity with the work that’s planned, and gives team members a chance to ask questions and remind each other about the steps involved or potential problems.
It’s typical to use the Fibonacci sequence for estimates: the bigger something is, the less we know, and it’s not worth quibbling over whether something is an 8 or a 9: it’s just big. An estimate over 5 is usually a sign that the task is too big and should be broken down into smaller parts.
Story points are used to calculate team velocity; the team’s velocity is based on an average of the points for tested and accepted stories over the last three iterations. We use the rolling velocity to determine the appropriate amount of work to take on in the next iteration. Story points should never be used to track individual team member productivity.
For a critical discussion of Agile practices including story points and estimation, see Best Practices? by Rebecca Sutton Koeser. For an alternate estimation scale, see Estimating Complexity, by Liz Keogh.