We develop a framework for rate-distortion theory and a theory of quantization for (sequences of) random variables of general distribution supported on general sets including manifolds and fractal sets. Manifold structures are prevalent in data science, e.g., in compressed sensing, machine learning, image processing, and handwritten digit recognition. Fractal sets find application in image compression and in the modeling of Ethernet traffic. Our contribution in rate-distortion theory is the derivation of a lower bound on the rate-distortion function that applies to random variables of general distribution and for continuous random variables reduces to the classical Shannon lower bound. The only requirement for our lower bound to apply is that the distribution of the random variable is absolutely continuous with respect to a sigma-finite measure of finite generalized entropy satisfying a certain subregularity condition. This condition is very general and prevents the measure from being highly concentrated on balls of small radii. Our contribution in quantization is the derivation of a lower bound on the n-th quantization error for random variables where the distribution is absolutely continuous with respect to a finite measure satisfying the above mentioned subregularity condition. To illustrate the wide applicability of our results, we evaluate these lower bounds for a random variable distributed uniformly on a manifold, namely, the unit circle, and a random variable distributed uniformly on a self-similar set, namely, the middle third Cantor set.