python (12.9k questions)

javascript (9.2k questions)

reactjs (4.7k questions)

java (4.2k questions)

java (4.2k questions)

c# (3.5k questions)

c# (3.5k questions)

html (3.3k questions)

Questions - aggregation

GroupBy based on condition Pyspark

I have a peculiar problem to which I don't see an easy solution (and efficient one). I have a list of addresses (more than 10M) with different user_time_active. I also have count per user_time_active ...
test-img

Martin Walczyński

pyspark

group-by

pandas-groupby

aggregation

Votes: 0

Answers: 1

Latest Answer

For anyone wondering the solution, I found one! End table looks a bit different, but it works! I use: df = df.withColumn("status", f.when(f.col("user_time_active") == 1, 1) \ ...
test-img

Martin Walczyński

Elasticsearch aggregation over several fields and push to an index

I'm having the following document in ElasticSearch, of the index data-ip "ipAddress": "ip-A", "timestamp": "Day 1, Month A", "value": 1 "ipAddres...
test-img

hoangtuansu

elasticsearch

aggregation

Votes: 0

Answers: 1

Latest Answer

You can use a terms aggregation with top hits aggregation, to achieve your required results { "size": 0, "aggs": { "ip-add-terms": { "terms": { ...
test-img

ESCoder

Merging column with array from multiple rows

I'm trying to merge the data from a dataset as follow: id sms longDescription OtherFields 123 contentSms ContentDesc xxx 123 contentSms2 ContentDesc2 xxx 123 contentSms3 ContentDesc3 xxx ...
test-img

DueSouth

arrays

apache-spark

pyspark

apache-spark-sql

aggregation

Votes: 0

Answers: 1

Latest Answer

You're looking for flatten function: x = df.groupBy("id").agg(flatten(collect_list("sms")))
test-img

blackbishop

While doing aggregation in MongoDB in lookup pipeline the id field is not available to perform match

As mentioned I want to match tenant id while looking in the logs collection and I am aggregating over Tenants collection. But this gives logs as an empty array. But when I did hardcoded the tenant_id...
test-img

shubham

database

mongodb

nosql

lookup

aggregation

Votes: 0

Answers: 1

Latest Answer

If you want to find all the logs where the tenant_id (in the log schema) equals the _id from the tenant collection you called the aggregation on, you can define this in the lookup itself. Example: con...
test-img

Maximilian Dolbaum

Posts

Questions

Blogs

Jobs