473,378 Members | 1,639 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,378 software developers and data experts.

Python/Numpy have I already written the swiftest code for large array?

I would like to get my script total execution time down from 4 minutes to less than 30 secs. I have a large 1d array (3000000+) of distances with many duplicate distances. I am trying to write the swiftest function that returns all distances that appear n times in the array. I have written a function in numpy but there is a bottleneck at one line in the code. Swift performance is an issue because the calculations are done in a for loop for 2400 different large distance arrays.*

Expand|Select|Wrap|Line Numbers
  1.  import numpy as np
  2.     for t in range(0, 2400):
  3.      a=np.random.randint(1000000000, 5000000000, 3000000)
  4.      b=np.bincount(a,minlength=np.size(a))
  5.      c=np.where(b == 3)[0] #SLOW STATEMENT/BOTTLENECK
  6.      return c
Given a 1d array of distances [2000000000,3005670000,2000000000,12345667,40007890 00,12345687,12345667,2000000000,12345667]
I would expect back an array of [2000000000,12345667] when queried to return an array of all distances that appear 3 times in the main array.

What should I do?
Dec 12 '17 #1
0 1386

Sign in to post your reply or Sign up for a free account.

Similar topics

2
by: Mark | last post by:
I am using PHP 4.3.5. I am having a problem in which my script is crashing when it is using a large array. I have simplified the problem down to the following example: <? set_time_limit(0);...
8
by: Pjotr Wedersteers | last post by:
Hello, I tried to create an array with 1000 cells, keys 0 thru 999 using $myarr = array (1000); But this leads to an array of 1 cell with value 1000; Now I have a workable solution for...
0
by: Patrick Mackeown | last post by:
Hi folks, I've a java RMI notBoundException when I try to use the web interface of a bought product. The client is inside the firewall and the server outside. They use the typical java RMI port...
2
by: sdeyoreo | last post by:
I'm getting an error:" http headers already written", when I redirect back the page I came from. I writing an on-line store app. The page 1 lists products, desc, price in a table. Click on a...
5
by: apm | last post by:
Any and all: Is there an efficient way to pass a large array from .NET to COM? Can references (or pointer) be passed from COM to NET and NET to COM without the object it refers to being copied?...
5
by: sylcheung | last post by:
Is it possible to use python to unit test C++ code? If yes, is there any example available? Thank you.
1
by: kgcsinte | last post by:
I hope someone can help with this one. This is a include file that is used to check if a user is logged in before displaying a page. I get this error when any page that references the include...
2
by: lbolla | last post by:
take a look at Python vs Matlab comparison for numerical computations: numerical-computing-matlab-vs-python+numpy+weave it seems that what stated in a previous thread was true! Python+Numpy...
4
by: Richard_Martineau | last post by:
Hello All I wonder if anyone can advise me or has done similar to the following? Basically I've downloaded the Python 2.5.2 source code that builds with Visual Studio 6.0. I've built Python...
10
by: pram29 | last post by:
Hello, I want to have a large array. I have a structure which contains 6 fields, 4 of them are unsigned char and 2 of them are unsigned short. I should have an array of 80 elements of type the...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.