
It's the male goldfinch that's a showy yellow-and-black bird. The female is the same size and shape, but is much duller-colored. You'll often see them together in pairs (they're monogamous) or groups. During the winter, the males will molt and take on the duller colors of the female. They'll grow brighter feathers again in the spring in time for mating season, which is the real purpose of all that color, after all.

Around August, the goldfinches start to regularly come to our yard to eat the seeds of our summer flowers. Among the flowers we offer, they seem to be particularly attracted to anise hyssop, purple coneflower, oxeye sunflower, and black-eyed susan. In the spring it also brings in the goldfinches when we let the kale and cilantro go to seed in the vegetable garden. Other flowers reported as attractive include milkweed, asters, and sunflowers. They'll also feed on grasses and the seeds of trees, including alder, birch, western red cedar, and elm.
Besides looking for the distinctive yellow markings, you can learn to listen for the song of the goldfinch. This video has a good example:
In the wild: Look for meadows on edges of woods. Goldfinches will feed on the seeds of the sun-loving flowers in the meadow (where you'll be able to see them better), but nest in the trees and shrubs.
In your yard: You can purchase birdfeeders specially designed for goldfinches, and fill them with thistle or nyjer seed. Squirrels seem less interested in these tiny seeds than in a typical birdseed mix, but be careful -- those plants may start growing where you don't want them from spilled seed. Goldfinches will also eat sunflower seeds out of a more typical birdseed. Or, just plant yourself a goldfinch garden with some of the native flowers listed above.
Like the photos in this post? Mouse over for credits; a click takes you to the photographer on Flickr.