You may not need to worry about copyright and fair use at all! Many works are not protected by copyright, or are already licensed to you or your institution for the uses you wish to make.
Unprotected Works/Public Domain
Copyright does not protect, and anyone may freely use:
See the Public Domain guide for more information about unprotected works.
The presence or absence of a copyright notice no longer carries the significance it once did because the law no longer requires a notice. Older works published without a notice may be in the public domain, but for works created after March 1, 1989, absence of a notice means virtually nothing.
Peter Hirtle from Cornell University created an excellent resource about copyright terms that explains the rules for determining whether a protected work is in the public domain. These rules are complex and somewhat hard to describe, partly because they changed many times during the 20th century. At their most basic, excluding anonymous works and works for hire, the rules can be summarized as follows:
Library-licensed works
Check your library's databases and catalogs. They may already have just what you need. You can also ask whether your institution has an institutional license with the Copyright Clearance Center.
Creative Commons licensed works
Learn to do effective searches for Creative Commons licensed materials. You may find exactly what you need with the rights you need to use it, available online for free. See the Creative Commons guide for more information.
Implied licenses
All of us who place materials on the open web do so knowing that people will use our works in certain ways (downloading, making personal copies, sending copies to friends, etc.). This is the essence of an implied license. I put my materials out there and even though I don't "expressly" give you the right to do these things, the law assumes that I must have intended to give you the right to do what a reasonable copyright owner would expect the public to do. Most nonprofit, educational uses would likely be within the scope of what people expect when they place materials on the open web. The scope of this license might be the same as or different from fair use, but it's good to know that we have both. Providing attribution should become automatic for you, whenever you use others' works.