Distributed computing and parallel computing are often considered similar even there is a slight difference between the two. Parallel computing implies a tightly coupled system connected with each other whereas the distributed system refers to a wider class of systems including those that are tightly coupled.
If we take a precise look at parallel computing, it refers to the model in which computation is divided among several other processors which are sharing the same memory. The architecture of parallel computing is characterized by the homogeneity of components. This means that each processor is of the same type with exactly the same capability as others.
The shared memory has single address space which is accessible to all the processors. Parallel programs are first broken down into several units of execution and after dividing, it is allocated to different processors which can communicate with each other by using shared memory. Parallel systems can be considered as architectures that featured multiple processes that share same physical memory and when combined considered as a single computer.
The example of parallel systems can be a cluster of nodes which are connected through an InfiniBand network that is configured with a distributed shared memory system.
Distributed computing on other hand encompasses any architecture or system in which computation is broken down into units and are executed concurrently on different computing elements, whether there are processors on the same computer, different nodes or cores within the same processor. Distributed systems comprise of a wider range of systems and applications that parallel computing and are often considered as a more general term.
Distributed computing implies that the location of computing elements are not the same and elements are completely heterogeneous in nature in terms of hardware and software features.
Example of distributed systems can be computing grids or Internet computing systems that combine together in architecture, systems, and applications.