Questions about distributed-computing
Read more about distributed-computing
python (12.9k questions)
javascript (9.2k questions)
reactjs (4.7k questions)
java (4.2k questions)
java (4.2k questions)
c# (3.5k questions)
c# (3.5k questions)
html (3.3k questions)
Issues running PySpark UDF with Databricks Connect
I'm having problems running my PySpark UDFs in a distributed way, e.g. via Databricks Connect.
For example:
import pyspark.sql.functions as f
class MyClass(object):
def __init__(self, number_stri...
Kasia Kulma
Votes: 0
Answers: 0
How to configure two servers with different IP in different networks into one cluster in Dask?
I'm trying to set up Dask cluster with docker containers of nodes in different networks.
Here is my simplified configurations with one worker. (public IP is randomly written)
Node configurations
1. No...
Dongjin Yoon
Votes: 0
Answers: 0
Limiting CPU resources of Ray
I'm trying to manage the resources of a remote machine that we use for a daily task (that uses Ray). Is it possible to limit the number of CPUs (or equivalently the number of workers) that Ray uses?
T...
M.Erkin
Votes: 0
Answers: 1
two pytorch DistributedSampler same seeds different shuffling multiple GPU-s
I am trying to load two version (original and principal component pursuit (PCP) cleaned version) of the very same image data set for training a modell using pytorch on a multiple GPUs remote machine.
...
Gergely Mathe
Votes: 0
Answers: 2