Baking soda (not to be confused with baking powder) is sodium bicarbonate (NaHCO3) that is added to baked goods to make them rise. Recipes that use baking soda as a leavening agent also contain an acidic ingredient, such as lemon juice, milk, honey or brown sugar.
When you mix together the baking soda, acidic ingredient and liquid you'll get bubbles of carbon dioxide gas. Specifically, the baking soda (a base) reacts with the acid to give you carbon dioxide gas, water and salt. This works the same as the classic baking soda and vinegar volcano except instead of getting an an eruption the carbon dioxide fizzes to puff up your baked goods. The reaction occurs as soon as the batter or dough is mixed, so if you wait to bake a product containing baking soda the carbon dioxide will dissipate and your recipe will fall flat. The gas bubbles expand in the heat of the oven and rise to the top of the recipe, giving you a fluffy quickbread or light cookies.
Waiting too long after mixing to bake your recipe can ruin it, but so can using old baking soda. Baking soda has a shelf life of about 18 months. You can test baking soda before adding it to a recipe to make sure it is still good.