Everyone sure likes to forget that Bismuth attacked a non-combatant.
More people like to forget that Pearl did it too.
I think an important distinction to make here is that sapphire was most likely a high-ranking member of BD’s court and Pearl was probably specifically there to poof and collect her and/or ask Blue Pearl to join them.
Bismuth just sorta punched a gem she saw, as far as we know. Lapis could be lying about why she was on earth but we don’t really have much reason to believe that she would lie.
Also, another important distinction to make is that THE BISMUTH THAT WE SEE IN THE FLASHBACK IS NOT THE BISMUTH WE SEE IN THE 30 MINUTE SPECIAL.
Lapis:
I was only meant to visit for a short time, but I got caught in the middle of the war. It was awful! I tried to run but… I was picked up by a Homeworld soldier and confused for a Crystal Gem and used as a tool.
In the episode Bismuth, it is revealed to us that Bismuths are not usually weapon smiths, but construction workers/artisans. At that time (as confirmed by centipeetle) construction work was underway for the Earth colony, which was most likely where Lapis was ordered to go, probably to record for the diamonds how the colony’s construction was performing (as we know she can easily display events she’s witnessed).
The diamonds never got Lapis’s report back because when Lapis arrived on Earth (at one of the construction sites, where a lot of homeworld-aligned bismuths would have been) she was mistaken for a crystal gem, and because she was a Lapis Lazuli, she was immediately imprisoned in a mirror because (as far as we can tell) the main function of Lazulis is not to manipulate water, it is to record everything they see and display it. Her capture would have been advantageous for homeworld had she been a Crystal Gem, because she would have been able to show her captors the exact location of any bases she had been to (they likely thought she was a spy sent to record information for the CGs). The Bismuth we know was likely already aligned with the CGs at this point, as Bismuth is called “One of the original Crystal Gems” by the CGs we know, and when Bismuth speaks of Rose, it sounds like they met on peaceful terms and discussed with each other.
tl;dr: FLASHBACK BISMUTH IS NOT CG BISMUTH AND PEOPLE REALLY LIKE TO IGNORE CANON DIALOGUE JUST SO THEY CAN HATE ON A CHARACTER