Michael Olise cancelled out an opener from Teden Mengi to leave the two teams level heading into the final exchanges, but Brown's first Premier League goal won it for Luton late on.
The result has seen the promoted outfit, who remain 17th in the Premier League table, move four points clear of the relegation zone, while Palace sit in 12th position on 15 points from their opening 13 matches.
Palace had a half-chance inside the first minute through Olise after smart work from Eberechi Eze, but his effort was always moving wide of the target.
Luton goalkeeper Thomas Kaminski made two excellent saves in the 22nd minute of the contest, first keeping out a long-range strike from Olise, before denying Jeffrey Schlupp from the rebound.
The hosts had a chance of their own in the 29th minute through Mengi, but the former Manchester United defender's strike was always comfortable for Palace goalkeeper Sam Johnstone.
Palace had seven attempts in the opening half, four of which were on target, but smart work from Luton's goalkeeper meant that the scores were level.
The strike from Mengi proved to be Luton's only attempt on target in the first 45 minutes, with the two teams heading down the interval at 0-0 after a period short of quality in the final third of the field.
Palace lost Eze to another injury early in the second period; Cheick Doucoure was then stretchered off the field with what looked to be a serious problem in a damaging few minutes for the Eagles.
The visitors had the ball in the back of the Luton net in the 67th minute through Odsonne Edouard, who finished at the second time of asking after a block from Tom Lockyer, but the goal was disallowed for handball following a VAR check, which sparked celebrations from the home supporters.
The stadium then erupted in the 72nd minute when Mengi made the breakthrough for Luton, finding the bottom corner with a smart finish after Palace had failed to deal with a corner.
Luton were only ahead for three minutes, though, as Olise levelled the scores with a brilliant goal, breaking into the home side's penalty box before picking out the far corner with a stunning strike.
The hosts incredibly scored a second in the 84rd minute through Brown, with the substitute managing to turn a cross from Chiedozie Ogbene into the back of the net from close range.
Kaminski made a smart save to deny Joachim Andersen late on as Palace looked for a second, and there were 12 minutes of added time at Kenilworth Road, with Olise also coming close late on, but Luton held on to secure a vital three points at a key stage of the campaign.
Luton will continue their league campaign away to Brentford next Saturday, while Palace will be aiming to bounce back from the disappointing result when they visit West Ham United next Sunday.