Michael Di Venuto (2) was dismissed early on when he was caught at mid wicket by Brown off Khan with Durham one wicket down with just six on the board.
Smith couldn’t repeat the innings of the first day and was dismissed in James Anyon’s first over when he was caught behind on 8.
Durham were 47-3 when Gordon Muchall was caught behind off Rana Naved for 10. Ben Stokes dug in and added 26 before he was trapped lbw by Panesar with the Durham total at 83-4.
Some big hitting from Ian Blackwell saw him reach his fifty in 44 balls which included three sixes and a boundary in two overs. He was caught at long leg by Brown on 51 off the bowling Anyon.
Michael Richardson joined Dale Benkenstein (24*) in the middle with eight overs remaining of the session and the pair ploughed on until the close of play having established a lead of 264 with 166-5 on the board.