There are a lot of factors involved here.
RedLiner740's points about the volumetric efficiency are bang-on. An engine is an air pump. So are a person's lungs. Imagine a person jogging, and that you vary the speed that the jogger runs by varying the amount that you strangle the jogger's neck. Doesn't sound very efficient, does it. Yet that's the way gasoline engines are controlled. They always want to make full power, but we don't always want full power, so we strangle them with a throttle.
So a small engine that can run near full throttle ought to be more efficient than an oversized engine at part throttle. Sometimes it's true, sometimes it isn't. The temperature of the internal components (primarily pistons and exhaust valves) comes into play when the engine is running near full load. Those temperatures will be highest when air/fuel ratio is close to the "ideal" mixture - no excess fuel, no excess air - in engineering terms, "lambda = 1". Most engines run somewhat rich (around lambda = 0.85 to 0.90) near full load, which brings the temperature down a little. Some run unnecessarily rich. Some other engines deal with the temperature by other means ... the lowly cbr125 engine has a ceramic-coated piston as original equipment! The new direct-injection engines (not yet used in motorcycles) can control the in-cylinder fuel distribution better rather than running wholescale rich. The bottom line is that the efficiency is usually best at moderately high load but just below the load where it starts running rich to protect itself. Whether a small engine running full load in rich protection-mode will use less fuel than a slightly bigger engine running less load and leaner air/fuel but with more pumping losses ... is a crapshoot. Sometimes yes, sometimes no. Depends which factor wins in a particular situation.
As for the twins versus fours ... The twins have bigger, heavier moving parts. The fours have more of them. Crapshoot as far as friction is concerned. But there's another factor - combustion efficiency. If you try to spin a twin-cylinder engine fast in order to make power equal to a four then it is going to need an extreme bore/stroke ratio. If you try to make that have high compression then you end up with a combustion chamber that looks like a pancake. That's bad for flame travel in the chamber - partly because it's further from the spark plug to the extremities, and partly because the flat-pancake chamber tends to kill charge turbulence, which is what helps flame travel. Slow combustion = lousy efficiency, and to make matters worse, slow combustion = high exhaust temperature = rich operation is needed at a lower threshold to protect the exhaust valves ... and there, folks, is why the big-twin sport bikes (RC51, TL1000, sports-model Ducati, etc) are often thirsty.
Back off the performance level a smidge - bring the revs down a bit to let the bore/stroke ratio return to sanity - and a twin can be very efficient. The BMW F800 series are fuel misers. My rental F800ST used 4.5 L/100 km with me beating on it in the Alps. The Honda NC700 series uses an engine that is more or less half of a Honda Jazz (Fit) car engine and it's supposed to be designed with fuel efficiency as a prime goal. At what point do you draw the line between performance and mileage? I was okay with the performance of the F800ST for what it was. I'm not convinced that the NC700 would do the trick for me, but I haven't ridden one.
Sometimes the OEM ECU calibration is out to lunch. That was the case with my ZX10R. I have no idea what they were thinking when they set that up. Leaning out the part-load made it run better in every possible way. I understand the need for rich-protection to avoid overheat in hot and heavy-traffic conditions ... but the ECU has the information it needs to only do that when necessary rather than doing it all the time.
cbr125 fuel consumption during normal riding 2.6 - 3.1 L/100 km
fzr400 4.2 - 4.5 L/100 km (carbs have been leaned out at part load to give me lean-cruise while still giving proper 0.85 to 0.90 lambda at full load)
zx10r 5.5 - 6.0 L/100 km (EFI set up with the same objective)
My car ... VW TDI diesel 5.5 L/100 km (diesel is a whole lot more efficient in general, for many reasons)