My first RPG was the second edition of Runequest, which I got when I was still at primary school - for my ninth birthday, I think. Runequest has a terrific combat system - so much so that other games often seemed a little flat in comparison. But Runequest combat is possibly too good - too detailed and too deadly - for an entirely satisfactory RPG experience. My abiding recollection of all those early games is that PCs were short-lived - and often short of a limb or two if they did survive.
Now, that sort of combat isn't necessarily a bad thing. The obvious implication is that PCs should seek to avoid combat where at all possible, because they never know when that puny trollkin will get lucky with a sling stone. Monster stats for the game tend to reinforce that; as virtually all creatures are playable as PCs, Runequest's standard villains don't have the default relative weakness that common-or-garden humanoid monsters do in D&D. The average broo is tougher and more dangerous than the average starting human PC, by dint of higher STR, CON and SIZ, natural armour and natural weapons (horns). So PCs have to be wary, at least until they're expert with their weapons and tooled up with magical items. Our characters never, ever survived that long.
Detailed combat of the Runequest sort also needs a lot of data. With its hit locations, variety of weapon proficiencies and damage modifiers, let alone the widespread access to magic, the average Runequest monster stats look little different from a character sheet. Preparing monsters used to take a lot of time, so there was a lot of reliance on the stat blocks in the Fangs supplement and other pregenerated stuff.
Things like Stormbringer and (I dimly recall) Call of Cthulhu simplified things by removing hit locations. That's a reasonable compromise, but you lose some of the grit and detail, which is half the point. All in all, I think Runequest is a great system for games that feature NPCs rather than monsters - the kind of game in which a single fight might be the climax of a session, or its centrepiece. Maybe something like The Unforgiven, or Kazuo Ishiguro's The Buried Giant: stories in which violence erupts only occasionally, but is swift and bloody when it does.
That sort of game sounds pretty good to me, but requires an alignment of approach of players and GM. And a lot of prep - just in case. And sometimes (and especially with kids involved), raiding monsters and stealing their stuff is precisely what's wanted.
That's where the D&D style - a pool of hit points that can be quite large, armour class and a feeling of relative safety for our heroes until their hit points run low - works much better. The game becomes a lot less about the odds (Will that trollkin get lucky? Probably not ...) and a lot more about resource management. And that's fun. In recent games I've run for kids and for adults, the strategies that arise when hit points run low have been exciting for the players and involving for the GM. If you're raiding a hobgoblin settlement to get as much plunder as you can, the ticking clock imposed by dwindling hit points and limited healing opportunities creates a nice bit of in-game tension. The Runequest equivalent - a raid on a trollkin gang as in The Rainbow Mounds - might be more dramatic, but in a sudden, messy and abrupt way.
So, while Runequest is far superior as a simulation of combat, the D&D approach can often make for a better game.
At the same time, most D&D systems could do better at simulating game combat without sacrificing the balance of the game. The most obvious example, I think, is the shield.
In medieval or ancient combat, a shield was a huge advantage. That's why most soldiers carried them. D&D doesn't reflect this very well: in BECMI, a shield gives a fighter a -1 adjustment to AC: half that of wearing leather armour. But in a fight, a man with a sword and shield is much better equipped than a man with a sword and leather armour. I don't many experts on medieval combat would disagree, and, historically, it's clear that equipping troops with shields generally took precedence over equipping them with armour. A man with a shield has a good chance of blocking a blow that would otherwise injure him; a man in leather armour (or, more plausibly in the real world, a padded jack or gambeson, has a chance that the blow will hurt him less. And if someone shoots a longbow arrow at you, a shield between you and it is going to be much more helpful than a few inches of thick cloth.
So, having shields as a small AC adjustment is underselling them significantly. It's also, I think, missing a trick for gaming. Shields should be a huge help for a fighter. But they're also big, bulky and awkward to carry around - especially when spelunking. That suggests a couple of things: bucklers would be a likely compromise (presumably less effective overall and especially against missiles, and possibly requiring more training to be used effectively - more like parrying with a weapon than instinctively blocking); and "proper" shields should often be involved in trade-offs - abandoned to allow more treasure to be carried, or to facilitate a quicker escape - albeit a riskier one, if combat ensues. There's a nice risk/reward dynamic in that, which I like, and it enhances the resource-management aspect of the game.
How could shields be done well in a D20 system? I suspect they ought to be separated from AC entirely - perhaps by giving a saving throw to be rolled simultaneously with the attack. And the chance to save should be high - maybe 50% or more. A fumble could lead to the shield breaking, perhaps - broken straps or split, or whatever. And a larger-than-human opponent might both reduce the save and increase the chance of damage. Bonuses for weapons that might disable a shield - pila or angons, perhaps - could be worked in too.
All this should have some interesting implications in a game. Fighters who are forced to abandon their shields because of potholing or cliff-climbing will feel exposed and are likely to seek replacements from slain foes. A kobold shield might work as a buckler for a human - and an orc shield should do just fine. So ordinary shields become treasure.
But what about heraldry? In a factional dungeon, you have to be a bit careful about which livery you're sporting. You might be able to persuade the hobgoblins of the Broken Fang that you're only carrying a shield made by the orcs of the White Eye because you've just killed some of them - but good luck with that. And if you walk back into the town you've just rescued from the ravages of the White Eye tribe? Well, we all remember Theseus, Aegeus and those black sails, right?