The sixth-tier outfit were seconds away from knocking out the Hammers, then seconds away from taking them to penalties, only to have their hearts broken twice at Aggborough Stadium.
Jarrod Bowen saved the Premier League side from humiliation after scoring with virtually the last kick of extra-time as the Hammers sealed a barely deserved 2-1 win.
They needed Declan Rice's injury-time leveller to force extra-time just as it looked like National League North side Kidderminster would pull off an almighty fourth round shock
Moyes admitted his side were second best and said he felt sorry for lowest-ranked side Kidderminster, who put in a brilliantly brave performance.
"It didn't look like it was going to go our way," the West Ham manager told talkSPORT.
"Great credit to Kidderminster to how well they've done.
"We gave them a little bit of a foothold early doors by giving them a goal, and you obviously don't want to come here and give the crowd and the public and media a chance to see there's a possible upset.
"Let's be fair, for long periods of the game that's how it looked."
He added: "I felt bad for Kidderminster because they earned the right to get to penalty kicks.
"They deserve an awful lot of credit for the way they did their job and made it difficult for us.
"In the last 15, 20 minutes we looked as though we were going to score.
"Whether we were going to get one, it seemed like it would be one of those days, but we got one.
"We gave them a wee bit of a chance to go in front and that made things a bit difficult as we had to chase it and it made it a bit more awkward," he said.
"They deserve all the credit. They played very well, made it very tough and we were fortunate to get through.
"They were unfortunate not to go through and at least take to penalty kicks."
Harriers, 115 places below the Premier League visitors, deserved so much more after nearly writing themselves into FA Cup folklore with one of its biggest upsets.