The model where multiple processors are attached to single memory block is PRAM Model. This model contains:
Independent operations can be performed by any number of processors in a particular unit of time. This facilitating in accessing the same memory location by different processors.
This problem is solved by enforcing the following constraints on PRAM model -
PRAM can be implemented in many methods. Some of the important methods are:
In this model more emphasis is given on data parallelism than on control parallelism. On different processors, multiple processes execute independently and a common memory space is shared. The rest of the processors can view any changes in the memory location.
As the same memory location is accessed by multiple processors it may lead to instances where the same memory location is accessed by more than one processor. One may read on the location and the other may write on the location at a time thus creating confusion. This can be avoided by implementing a control mechanism called lock / semaphore.
Shared memory programming has been implemented in the following −
Examples of thread libraries include − SolarisTM threads for Solaris, POSIX threads as implemented in Linux, Win32 threads available in Windows NT and Windows 2000, and JavaTM threads as part of the standard JavaTM Development Kit (JDK).
The control level on the memory system is low on the concept of shared memory but is tedious and erroneous. It is suited for system programming than application programming.
The common parallel programming approach used in the distributed memory systems is Message passing. The parallelism is determined by the programmer. All the processors contain their own local memory unit and use a communication network for exchanging data.
Message-passing libraries are used for communication. In addition to data, the following components are included in the message:
The following methods are used by the processors for communication -
The simplest form of message passing is point-to-point communication. The sending processor sends the message to the receiving processor by using different modes of transfer -
Message passing is done by using more than two processors under Collective communication. Collective communication is done in the following modes:
There are three types of broadcasted messages -
Several message passing libraries are available but out of which the following two are relevant -
MESSAGE PASSING INTERFACE (MPI)
For providing communication among all the concurrent processes in the distributed memory system, the universal standard is MPI. At least one implementation of the message passing interface is provided by most of the parallel computing platforms. It is being implemented as collection of predefined functions called library and are called from the languages such as C, C++, Fortran, etc.
Merits of Message Passing Interface
Demerits of Message Passing Interface
PARALLEL VIRTUAL MACHINE (PVM)
The message passing system that connects the separate host machines to form a single virtual machine is PVM, also known as single manageable parallel computing resource. Routing of message, conversion of data, scheduling of tasks in the network are managed by PVM.
Features of PVM
The operations on the data set are performed simultaneously by this data parallel programming model. On the same data structure, operations are collectively performed by the processors. Each task is performed on a different partition of the same data structure.
But all the algorithms cannot by specified in terms of data parallelism. And hence data parallelism is not universally accepted.
Data decomposition and data mapping to the processes is specified by the data parallel languages. It contains the data distribution statements which enable the programmer to control the data. For instance, which data will go into which processor for reducing the cost of communication.
Parallel Algorithm Related Tutorials
|Data Structure & Algorithms Tutorial|
Parallel Algorithm Related Practice Tests
|Data Structure & Algorithms Practice Tests|
All rights reserved © 2020 Wisdom IT Services India Pvt. Ltd
Wisdomjobs.com is one of the best job search sites in India.