Hfrxdjt

Visualizing the difference curve in a 2D plot?

El Dorado Word Puzzle II: Videogame Edition

Why would five hundred and five be same as one?

Can I cause damage to electrical appliances by unplugging them when they are turned on?

The Digit Triangles

If the only attacker is removed from combat, is a creature still counted as having attacked this turn?

Why is the Sun approximated as a black body at ~ 5800 K?

When is "ei" a diphthong?

Why can't the Brexit deadlock in the UK parliament be solved with a plurality vote?

What is the meaning of "You've never met a graph you didn't like?"

How would you translate "more" for use as an interface button?

How do I prevent inappropriate ads from appearing in my game?

Limit max CPU usage SQL SERVER with WSRM

What should be the ideal length of sentences in a blog post for ease of reading?

Check if object is null and return null

Is there a reason to prefer HFS+ over APFS for disk images in High Sierra and/or Mojave?

How do I fix the group tension caused by my character stealing and possibly killing without provocation?

Should I warn new/prospective PhD Student that supervisor is terrible?

Overlapping circles covering polygon

When and why was runway 07/25 at Kai Tak removed?

Can you identify this lizard-like creature I observed in the UK?

Air travel with refrigerated insulin

Quoting Keynes in a lecture

PTIJ: Which Dr. Seuss books should one obtain?

Building a large distance matrix

Image clustering by similarity measurement (CW-SSIM)Distance measure for ternary featureDistance measure calculation addresses for record linkingAgglomerative Hierarchial Clustering in python using DTW distanceHow to build an encoder using a distance matrixDistance between very large discrete probability distributionsClustering time series based on monotonic similarityClustering algorithm for a distance matrixPositive semidefinite kernel matrix from Gower distanceClustering based on distance between points

0

$begingroup$

I am trying to build a distance matrix for around 600,000 locations for which I have the latitudes and longitudes. I want to use this distance matrix for agglomerative clustering. Since this is a large set of locations, calculating the distance matrix is an extremely heavy operation. Is there any way to opimize this process while keeping in mind that I am going to use this matrix for clustering later. Below is the code I am using.

from scipy.spatial.distance import pdist
import time
start = time.time()
# dist is a custom distance function that I wrote
y = pdist(locations[['Latitude', 'Longitude']].values, metric=dist)
end = time.time()
print(end - start)

python clustering distance

share

asked 4 mins ago

Karthik KatragaddaKarthik Katragadda

113

New contributor

Karthik Katragadda is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

0

$begingroup$

I am trying to build a distance matrix for around 600,000 locations for which I have the latitudes and longitudes. I want to use this distance matrix for agglomerative clustering. Since this is a large set of locations, calculating the distance matrix is an extremely heavy operation. Is there any way to opimize this process while keeping in mind that I am going to use this matrix for clustering later. Below is the code I am using.

from scipy.spatial.distance import pdist
import time
start = time.time()
# dist is a custom distance function that I wrote
y = pdist(locations[['Latitude', 'Longitude']].values, metric=dist)
end = time.time()
print(end - start)

python clustering distance

share

asked 4 mins ago

Karthik KatragaddaKarthik Katragadda

113

New contributor

Karthik Katragadda is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

add a comment | 

$begingroup$

I am trying to build a distance matrix for around 600,000 locations for which I have the latitudes and longitudes. I want to use this distance matrix for agglomerative clustering. Since this is a large set of locations, calculating the distance matrix is an extremely heavy operation. Is there any way to opimize this process while keeping in mind that I am going to use this matrix for clustering later. Below is the code I am using.

from scipy.spatial.distance import pdist
import time
start = time.time()
# dist is a custom distance function that I wrote
y = pdist(locations[['Latitude', 'Longitude']].values, metric=dist)
end = time.time()
print(end - start)

python clustering distance

share

asked 4 mins ago

Karthik KatragaddaKarthik Katragadda

113

New contributor

Karthik Katragadda is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

$endgroup$

from scipy.spatial.distance import pdist
import time
start = time.time()
# dist is a custom distance function that I wrote
y = pdist(locations[['Latitude', 'Longitude']].values, metric=dist)
end = time.time()
print(end - start)

share

asked 4 mins ago

Karthik KatragaddaKarthik Katragadda

113

New contributor

Karthik Katragadda is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

share

asked 4 mins ago

Karthik KatragaddaKarthik Katragadda

113

New contributor

Karthik Katragadda is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 4 mins ago

Karthik KatragaddaKarthik Katragadda

113

New contributor

Karthik Katragadda is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

asked 4 mins ago

Karthik KatragaddaKarthik Katragadda

113