Alonso and converted cornerback Aaron Williams each had two interceptions in helping the Bills hang on for a 23-20 win over the Ravens.
Alonso's second interception with 57 seconds left sealed the win, when he made a diving catch to intercept a tipped pass intended for Dallas Clark over the middle.
Fred Jackson had 87 yards rushing, and scored on a 14-yard run for the Bills, who are off to a 2-2 start under rookie head coach Doug Marrone. C.J. Spiller added 77 yards rushing in helping Buffalo combine for 203 yards on the ground against a defense that had allowed just 224 in its first three games.
Joe Flacco threw a career-worst five interceptions, and finished 25 of 50 for 347 yards passing, and two touchdowns.
Torrey Smith had five catches for 155 yards, including a 26-yard touchdown for the Ravens (2-2).
Alonso, a second-round pick out of Oregon, has four interceptions on the season. And he became the team's first rookie linebacker to have an interception in three consecutive games.
Defensive tackle Marcell Dareus had two of Buffalo's four sacks. The Bills' defense bounced back from a porous outing in a 27-20 loss to the New York Jets last week.
The five interceptions were the most by the Bills defense since it had six in a 16-13 overtime win over the New York Jets on Oct. 18, 2009.
The Bills limited the Ravens to 345 yards, after giving up 513 to the Jets.
And they did it with a defensive secondary that was missing both starting cornerbacks Stephon Gilmore (broken hand) and Leodis McKelvin (right hamstring) and starting safety Jairus Byrd (foot injury). The defensive backfield was thinned even further after Williams was unable to finish the game because of a sore back.
Rookie quarterback EJ Manuel went 10 of 22 for 167 yards, including a 42-yard touchdown pass to fellow rookie Robert Woods. Manuel, however, threw two interceptions and also lost a fumble.
The Bills never trailed after Woods scored to make it 13-7 midway through the second quarter.