Some fleets in EVE are open to anyone. These could be loose, improvised groups for mining or running missions, but they might also be "NPSI" fleets seeking PvP.
Some public fleets are mechanically public in game, visible in the Fleet Finder. Others are equally open to all comers, but are coordinated using public in-game chat channels for "x-ups" and public voice chat in programs such as Discord for voice communication.
Safety considerations
Plenty of public fleets are safe, above-board groupings. Those advertised through reputable community hubs are especially reliable. Players should, though, exercise caution about taking unexpected fleet invites from other players they don't know.
Being in a shared fleet with someone else who is above you in the fleet's internal hierarchy can let them warp your ship with them, if they're "on grid" (in close proximity) to you. If your safety settings are orange or red, someone in a shared fleet with you and very close to your ship can use a filament to teleport both of you, after a spool-up timer, to another system, to Pochven, or into Abyssal Deadspace. All of these abilities can be very handy in the hands of trusted fleetmates, or dangerous in the hands of untrustworthy ones.
However, sharing a fleet with someone does not let them jump your ship through gates or wormholes, or along acceleration gates, as all of these require individual pilot action. Being in a fleet with someone else does not remove the CONCORD penalty for violence in highsec systems; someone who tells you otherwise and suggests firing on their ship while in a shared fleet is probably trying to get CONCORD to blow you up.