There is no such thing as "self sufficient". That's not what homesteading is about. Homesteading is a part time job for someone that has a family business or a very flexible work schedule. Have a 9 to 5 job that you MUST be at? Homesteading is not for you. That probably means that Homesteading really is not for folks in debt. If you are sweating a big mortgage, I don't see how you cut it Homesteading. This is not a way to get rich, this is a way to become more self-reliant. I think any mortgage at all makes it untenable. That means save a lot of money and/or buy a homestead with modest improvements.
In order to grow most of your own food you will need at least 5 productive acres, a small barn - large enough to hold a winter's worth of hay and hold your animals for working, whatever that may be - a full acre garden as a minimum and maybe up to all 5 acres in cultivation if you want a reasonable cash crop, but lets talk about what you will need to produce to provide most of your own food and grow or raise enough stuff to trade for feed or seed or what have you.
You do not need breeding animals necessarily. Raising one, or perhaps a couple of weanling pigs (you can buy them for $35 to $50 each when they are about 20 lbs) takes about 6 or 7 months until they reach slaughter weight and will supply all of the pork your family could ever eat in a year (about 180 lbs of dressed meat from each 250 lbs hog; a family of 5 with 3 teenagers would need to raise 2 hogs, and roughly half of their feed will come from kitchen refuse - hogs are the ultimate in recycling). Lard as a cooking oil is an option if you raise pigs (I didn't say you HAD to do that, or that I do, only that you have the option).
A herd of 6 goat does and 2 bucks (just in case one dies) will give you all of the milk, cheese, and goat meat you can stand (and you won't have to weed or cut the lawn if you move them around right by picketing them, especially if they have been bottle raised... but, be careful not to let goats get loose - they will HAPPILY kill every fruit tree, garden plant, and decorative bush you have in the blink of an eye. Each goat will only consume $60 per year in feed IF you have an acre of pasture (for the 8 goats) and decent rainfall. Droughts change the picture somewhat. We spend ZERO on vet bills for goats - we vaccinate and worm them ourselves, and do our best if one gets sick but we do not spend money for healthcare for goats. You shouldn't lose many goats if your slaughter rotation is correct (livestock lifecycle management is a science as well as an art. Which animals you choose from a herd, and when you choose them, for slaughter affects the future production of the herd. It is REALLY, REALLY dumb to lose (that is, died) an animal that you have fed and cared for rather than slaughter it to feed your family). If the herd becomes outsized, dairy does can always command a decent price in the market place, meat goats less so... perhaps $1 per pound on the hoof, which would barely cover your gas money in delivering them.
Milking is WORK. You have to do it everyday at roughly the same time. I don't like to milk twice per day... so I leave the kids on the nannies and only milk in the evening. We get less milk per nannie, but far more than we can use. Another thing... most nanny goats will go dry for 2 to 4 months per year, usually at the same time, so you won't have fresh milk all year round with goats. Either get a cow, or freeze the goat milk, or go to the store.
Goats are FANTASTIC for improving pastures in an organic environment. Their hooves do no damage, and they eat broad leaf weeds, which cows, horses and even sheep will not eat. This takes competitive pressure off of the grasses in your pasture, and their manure does not kill a couple square feet of pasture with each "flop" as is the case with cows - in fact their manure is evenly distributed and fertilizes the pasture nicely.
Raising "Bottle Calves", unwanted bull calves from dairy farms and such, can be purchased for about $100. If you have enough grass, that with $50 of calf milk replacer will get you a 1,000 lbs steer in about a year which will yield 600 to 650 lbs of meat. Usually that is too much for a family (unless you have a bunch of teenage sons...), but it is not hard to find another family to sell a side of organic, grass raised beef to go in with you on the steer.
Chickens are a must. 50 dual purpose chickens and 5 roosters left to free range - you will still have to provide some feed, and of course water - will supply all of the eggs and poultry meat a family of 5 can consume, provided you have an incubator that can handle at least 50 eggs. I let our birds free range all year except spring and early summer (at which time I use chick tractors) because at that time of year they are a danger to my garden and I want to collect all of their eggs (free ranging saves on feed but you won't find all of their eggs... but you will find most of them once you know where their favorite nesting places are) to go into our incubator (I have a 280 egg commercial incubator and I will hatch out 500 to 600 chicks per year). I use them to trade and as gifts and keep about 150 each year. 50 meat roosters, 50 chickens for next year, and the balance for loss and predation.
Dual purpose birds like the Rhode Island Red will lay 5 to 6 eggs per week in June, but only 1 or 2 per week in December. You will have too many eggs during spring and summer, even if you have a big incubator, and that's ok... the hogs are only too happy to consume the excess eggs, and your neighbors will never be offended to find a box of fresh eggs at their doorstep.
Feed and hay are inexpensive if your slaughter rotation is managed properly, and that means keeping few animals over the winter... so you don't have to grow all of your feed. If you live in the country like we, it won't be hard to find a farmer to trade pork for corn...
The Garden.
There are kitchen gardens... and then there is the homesteader's garden. Kitchen gardens grow a couple of tasty tomatoes... I grow 400 lbs of potatoes and sweet potatoes per year, and a similar, if smaller by weight, volume of sweet corn, field corn (for corn meal and flower), beans, and peas. I will grow 50 broccoli heads (they freeze well), 25 to 35 heads of cabbage (think slaw), 50 to 100 roma tomato plants (for canning), onions, garlic, spinach, turnips (for root and greens), cucumbers for salads and pickling, peppers (hot and sweet), carrots, lettuce, squash, zucchini, artichokes, asparagus, rhubarb, watermelon, cantaloupe, blueberries, strawberries. Any thing that goes bad, is infested, or we have too much of goes to the hogs. Some folks play golf; I like gardening.
I have peach, pear, plum and apple tress from which we get barely enough fruit for the family... I should have planted more trees. It takes 3 years to get any production out of these tress, and then only if you buy yearling or 2 year old trees (I like the dwarf varieties). 6 trees of each is the bear minimum.
I don't grow wheat, I buy it. I am not a "locavore". I love bananas with my breakfast, citrus all day long, cook with olive oil and use it like most folks use butter. I LOVE coffee in the morning.
Most of your food calories will be in the form of grains, meat, dairy, eggs, and potatoes/beans... vegetarians and vegans would simply not be able to get enough calories out of their diet to do the kind of work that goes on at a homestead IMHO (this is when I get the "farmers in India don't eat meat" routine... usually from somebody who couldn't find India on a map). Also, organic farming/gardening requires manure - LOTS of manure - and manure comes from animals, and if you don't eat the excess (males) animals they will eat everything - including your garden.
Back to manure... We literally have TONS of the stuff accumulate under and around the hay rings during the winter, and pay a neighbor to use a front end loader to scoop it up and dump it in the garden. A better approach for the small holder might be to fence the animals in the area you intend to cultivate or garden during the winter. One of my neighbors plants his corn where ever the cows were wintered.
You can expect to spend $7k to $15k to outfit your place with all of the tools, materials, fencing, and livestock you need to get started. Yes, a piglet only costs $50... but you need a pig pen and that might run you $150 (they last decades), and feed ($75 - $100 per hog till slaughter), and a "coffin freezer" ($500) to keep your meat. Laying hens are $10 each - and you need 50 (alternatively, you can buy day old chicks for $2.50, feed them and wait 5 months for them to start to lay eggs...), and they need a coop. Raised beds don't just show up in your garden - you need blocks, or pavers, or garden timbers, as well as shovels, picks, tines, hoses, etc.... and the dirt doesn't jump into those beds, either, and the weeds don't jump out on command - expect to do some work. You will need to maintain barns and sheds if you have them, and build them if you do not. You will need a chop saw, a table saw, and a bunch of hand tools...
On the other hand... we NEVER have to "run to the store" to buy milk, eggs, and bread. In fact, we go days without moving a vehicle. The savings on gasoline and food purchases make the initial investment an excellent bet. The amount of time one spends driving around doing "errands" (and harming the environment, killing animals crossing our roads, risking your life and the lives of others) begins to pile up in earnest. Some people respond well, taking up hobbies and intellectual pursuits. Others without this capacity might say they are going stir crazy. That's up to you.
I have not, as yet, installed a green house. That is next. I do collect and save seeds, and seed potatoes and sweet potatoes. Its probably cheaper to just by the started plants... but I like to do it from scratch just for practice and fun.
A quick summary: 1 or 2 meat piglets purchased each spring and slaughtered in late fall. 1 bottle calf to be slaughtered at 12 to 18 months (there is always a risk that an animal dies, and if you have only one....), 8 goats (6 female, 2 male), 50 hens and 5 roosters, one mother of a garden, and 6 fruit trees of each sort (at a minimum). This will provide 100% of the meat, milk, and eggs, and over 50% of the fruit and vegetables for a family of 5. This will require a burst of effort and time each spring and fall (preserving is bigger than harvesting), but it will not be a full time job by any stretch - unless you want it to be. Of course, you can expand your efforts and leverage your time by raising more of one or all and marketing your production.
Oh, and time... 3 or 4 growing seasons before you start to put it all together, make your contacts and establish relationships, repair the damage the previous guy did to pastures or lack of maintenance on out-buildings, make mistakes, learn skills like food preservation, animal husbandry and care, and provision all of your hardware, tools, and equipment. It is doable, just not by reading books like "Gardening When It Counts". You have to actually do it, and its a lot more than gardening.