The library has its beginnings in the California State Normal School, an institution founded in San Francisco in 1862 as a teacher's college. Once the college relocated to San Jose in 1870, the library always had a dedicated room at the college, but had to be moved several times due to natural disasters that damaged the buildings. The library was housed in the first Normal School Building from 1872-1880, the second Normal School Building from 1881-1906, "The Shacks" (temporary buildings used after the 1906 earthquake) from 1906-1910, and the original Tower Building from 1910-1942.
The library continued to rapidly expand, especially after the California Normal Schools were re-designated Teachers' Colleges in 1921, and then State Colleges in 1935. The library in the Tower Building was expanded in 1942, and the Library Annex was built in 1956. As the library needed additional space, the old Carnegie Library Building (originally the public library, and then the student union) was demolished to build Library North.
After San Jose State College became a university, it began planning a multi-story library building, the Clark Library, which was finished in 1982. Because the building was too small, the collection was split between the Clark Library and Wahlquist North (formerly known as Library North, one of the buildings in the old library complex). The Wahlquist complex buildings were torn down to build the Dr. Martin Luther King, Jr. Library, which opened in 2003. In a unique partnership, the King Library houses both the San Jose State University library and the San Jose Public Library main branch.