Stocks are certificates that entitle the holder of the stock to a proportionate share of ownership in a company. For example, if there are 100 shares of stock available from a company and an investor owns 10 shares, the investor owns 10% of the company. For publicly traded companies, investors hold their shares with a brokerage rather than actual certificates of paper.
Companies do not need to be public to issue shares of stock. Private companies can issue shares as well, although private shares are far less liquid than public shares. Companies generally issue stock to raise money for their business.