Parallelisation using numba by calling type of a self made class


#1

I have never used numba before and I search a solution to parallelise a python code on GPU without rewriting all of my code.
There are two classes made by myself and called surface and system. Rougly speaking, system is a list of surfaces.
The function to parallelise is called trace() and belongs to class system.
Each computation to paralellise use the tensorflow functions surface.sag_param and surface.champsVec defining the current surface.
It loops on system.posx and system.champsx and inside this two loops it does a serial computation over the surfaces (list of object of class surface).
This computation is quite generic and I would want to conserve this flexibility.
I wounder if it is possible to do this paralellisation without rewrite all the code inside my classes ?
I just tried to write @jit(npython=True,parallel=True) before the function trace() and replace range() by prange() (for numCHamp and numRay, but I get the error :
TypingError: Failed at nopython (nopython frontend).
Untyped global name system : cannot determine Numba type <class ‘libsystem.system’>
Indeed numba does not know this type.
By setting the parameter nopython to False, I get the error "TypeError : can only concatenate tuple (not “dict_values”) to tuple
Here is the shape of my code (the two classes with the function system.trace() to paralellise).
Thank you for your help.

class system :

def __init__(self,**kwargs):
    self.numSurfaces = 3 # nombre de surfaces (Surface 0 : surface fictive où on défini les rayons incidents, Surface 2 : surface fictive où on regarde les rayons émergeants )
    self.surfaces = ... #list of elements of class surface
    
    self.champsx = ... #1D numpy array 
    self.champsz = ... #1D numpy array 
    self.posx = ... #3D numpy array  
    self.posy= ... #3D numpy array          
    self.numSurfaces = len(self.surfaces) 
	
def trace(self):
	# function to parallelize with respect to numChamp and numRay
    
   #parallel computation possible
    for numChamp in range(len(self.champsx)):
	#parallel computation possible
        for numRay in range(self.posx.shape[0]):
			#computation of a point and a direction on each  surface
            surf = self.surfaces[0]
            P = np.array([self.posx[numRay,numChamp,0],self.posy[numRay,numChamp,0],\
			surf.sag_param(self.posx[numRay,numChamp,0],self.posy[numRay,numChamp,0],*surf.params.values())])
            Q = np.array([np.sin(self.champsz[numChamp])*np.cos(self.champsx[numChamp]),\
			np.sin(self.champsz[numChamp])*np.sin(self.champsx[numChamp]),np.cos(self.champsz[numChamp])])
            
			#sequential computation
            for numSurf in range(1,self.numSurfaces):
                surfPrev = surf
                surf = self.surfaces[numSurf]                    
                P = surf.transferPoint(P,Q)                

                if numSurf<self.numSurfaces-1:
                    Q = surf.transferDirection(P,Q)
					
                self.posx[numRay,numChamp,numSurf] = P[0]
                self.posy[numRay,numChamp,numSurf] = P[1]    

class surface:

def __init__(self,**kwargs):        
    
    self.params = ... # a dictionnary containing some parameters value              
    self.sag_param =  ... #a tensor flow function depending on self.params and 2 other spatial variables          
    self.champsVec =  ...# idem with self.sag_param


def bz(self,u,v,cx,conic,cy,conicy,coefsZernike):
   
def transferPoint(self,point,direction):
    #a transfer function returning a point and using sag_param and params of self

def transferDirection(self,point,direction):
	#a transfer function returning a direction and using champsVec  of self

#2

The answer is pretty much a straight no, I’m afraid. You can’t in general apply jit to methods, for starters.

But even if you pull apart that class, it still won’t work because the jit engine can only compile code when it acts on simple and familiar types — which, in essence, means integers, floats, bools, and basic container types and numpy arrays.

See http://numba.pydata.org/numba-doc/latest/reference/pysupported.html


#3

Thank you John, I will try to rewrite my code by following this documentation.
Do you think that numba is the best way for beginners like me to parallelise a code on GPU ?
Many thanks


#4

You’re welcome @drogoua

I love numba but it does take some time to understand it’s limitations and how to use it best.

If you’re looking to implement machine learning routines, then I believe tensorflow already implements parallelization under the hood in some of its routines, and that parallelization can target the GPU. But I don’t have experience using it myself.

Good luck with your studies.

John.