According to The Global Water Cycle:

The total amount of suspended particulate phosphorus (inorganic and organic) transported by rivers at present is approximately 20 Tg P/yr (Meybeck 1982). Part of this is from the erosion of phosphate minerals in rocks, primarily apatite, and much of the rest is from human effects---increased soil erosion and transport due to deforestation and agriculture, and increased organic matter removal due to agriculture and fertilization. (Berner 1987)

The rises and falls on the graphs can once again be accounted for by considering the causes for the changes in nitrate nitrogen levels, precipitation and fertilization being the primary explanations.