You have a cleanout somewhere, unless it's been removed or covered over. Find the location of your main stack, that's the 3 or 4" pipe that runs vertically thru the basement floor. If your house was originally on a septic, it will run horizontally out thry the basement wall2-6' below grade. There will be a large Y fitting and a cap within a foot or so of where it goes thru. If it's ABS, use a pipe wrench to unscrew the cap. If it's cast, you need to use penetrating oil, then a hammer on the tabs to unscrew.
If you're rooted up or greased and rooted up, you'll need a root cutter head for the snake. Ask for one at the tool rental place or buy a cheepie off amazon or HD for about $30. (you don't need a pro cutter head, the cheap ones work for about 10 cleanings). Clip the head on at the end of the snake then feed the running snake slowly down the cleanout.
You will have trouble using a cheepie $30 boroscope on a 4" drain, it will curl up in a few feet (think pushing a shoelace) , a proper drain scope costs $300 or more. If you have a boroscope with a 15m line, you can zip tie it to the snake as long as you can hand feed the snake thru the drain.
You can clear roots, but you can't stop them from coming back. I'll use foaming root killer if the drain is still flowing (not completely clogged) , it's a $30 temp fix (works 90%of the time for me) - then it's a once a year application. Just pour into your cleanout, or pop off your toilet nearest the basement and pour it into the flange. DO NOT POUR IT DOWN THE TOILET OR ANY DRAIN THAT HAS A TRAP. You need 12 hours min with no drain activity, leave it overnight, no flushing, showers or water in the drains at all, make sure no water flows to the drains from your furnace, AC or trap primers.