Best Practices for Scientific Computing

Standard

A summary of a very interesting paper on “Best Practices for Scientific Computing” I read a year ago.

andrea cirillo's blog

I reproduce here below principles from the amazing paper Best Practices for Scientific Computing, published on 2012 by a group of US and UK professors. The main purpose of the paper is to “teach”  good programming habits shared from professional developers to people  that weren’t born developer, and became developers just for professional purposes.

Scientists spend an increasing amount of time building and using software. However, most scientists are never taught how to do this efficiently

Best Practices for Scientific Computing

  1. Write programs for people, not computers.

    1. a program should not require its readers to hold more than a handful of facts in memory at once
    2. names should be consistent, distinctive and meaningful
    3. code style and formatting should be consistent
    4. all aspects of software development should be broken down into tasks roughly an hour long
  2. Automate repetitive tasks.

    1. rely on the computer to repeat tasks
    2. save recent commands in…

View original post 226 more words

Advertisement

Assorted links – Data Science with R

Standard

last updated: 2015-08-29

References & Most helpful commands

Tutorials & Handy packages

Hands-on dplyr tutorial for faster data manipulation in R Interactive Visualizations From R Using Rcharts rMaps – Interactive Maps from R (github repo) (requires “devtools” from cran)
Using R for Psychological Research – Personality Project, William Revelle
DataCamp courses
Try R by Code School (on codeschool)
Introduction to R, Leada

Visualization Packages

see Assorted links – Data Visualization (to be published later)

Papers

Tidy Data, Hadley Wickham [PDF]

Journals

Big Data & Society – Open-access journal

Hacks for better productivity

Sublime and R

Using Sublime Text 2 for R Using R in Sublime Text 3

Books

Video (training) courses

Introduction to Data Science with R, Garrett Grolemund, O’Reilly Media

Lists of Resources by others

Data Mining

Scraping Twitter and Web Data Using R – Pablo Barbera

Numerical Analysis
Interoperability
Data Sources

see Assorted links – Data sources (To be published later)

If you’d like to contribute to this list, please leave them in the comments below.

NASA’s JPL releases Two Math Libraries

Standard

NASA’s Jet Propulsion Laboratory (JPL) has released two large math libraries under an open source (BSD) license. Via Degenerate Conic:

“MATH77 is a library of Fortran 77 subroutines implementing various numerical algorithms. It was developed over decades at JPL, and contains some very high-quality and time-tested code. The code is released under a BSD-type license. There is also a C version for people who love semicolons.”

This goldmine includes basic mathematical functions, random number generators, linear algebra routines, solvers for systems of nonlinear equations, curve fitting, interpolation, and quadrature routines, and much more. The libraries are available at Netlib and are accompanied by 619 pages of detailed documentation.

[via jblevins.org]

Fortran Wiki

Wolfram Language Overview

Link

Wolfram Language Overview

Mathematica & the Wolfram language

Standard

“We call it the Wolfram Language because it is a language. But it’s a new and different kind of language. It’s a general-purpose knowledge-based language. That covers all forms of computing, in a new way.
There are plenty of existing general-purpose computer languages. But their vision is very different—and in a sense much more modest—than the Wolfram Language. They concentrate on managing the structure of programs, keeping the language itself small in scope, and relying on a web of external libraries for additional functionality. In the Wolfram Language my concept from the very beginning has been to create a single tightly integrated system in which as much as possible is included right in the language itself.”

This is what Wolfram himself mentions in his blog post last November “Something Very Big is Coming: Our Most Important Technology Project Yet” [1] as an introduction to another blog post which appeared on November 21 “Putting the Wolfram Language (and Mathematica) on Every Raspberry Pi” in which he publicizes his plans and futuristic but old views for Mathematica and the Wolfram language [WL].

The projects that Wolfram uncovered are:

  • Mathematica and Wolfram language on the Raspberry Pi [2]
  • Mathematica Online [3]
  • Wolfram Cloud [4]

Even though many would argue against this vision of a language that does not rely on external libraries or modules, including my self, given that libraries is a way to reuse code by other programs, which in the case of the WL is not the way to go given that it is mostly a commercial software, I think that what Wolfram is attempting is the way to further increase the value and power of the WL and Mathematica as indicated in this quote:

“Seeing how incredibly powerful it is to be able to treat code like data, interspersing little programs inside a piece of graphics, or a document, or an array of data. Or being able to put an image, or a user interface element, directly into the code of a program. Or having any fragment of any program immediately be runnable and meaningful.”

“In most languages there’s a sharp distinction between [algorithms and data]. Not so in the Wolfram Language… Data becomes algorithmic. Algorithms become data.”

I truly appreciate this thought that algorithms and data could be similarly dealt with. This is the first step towards a very powerful language that is, I think, capable of being used for artificial intelligence on a large scale.

What Wolfram publicized in these blog posts further supports my belief in this computer algebra system [CAS]. Moreover I see the WolframAlpha knowledge-based engine [5] as the first draft, which lacks (to my knowledge) the ability to display web results like other engines, of a full-blown search engine that will replace many of the current ones.
Further, introducing a free version of Mathematica and WL bundled within the Raspberry Pi will serve well both the Raspberry Pi users and Wolfram [6] it self.

Moreover, Wolfram intends to connect everything you can imagine to the internet [7] and this is why Wolfram started the “Wolfram Connected Devices Project” [8].

This post was written a few weeks before the following video was posted in which Stephen Wolfram talks about the WL. I leave you with this video.

Footnotes

[1] Something Very Big Is Coming: http://blog.stephenwolfram.com/2013/11/something-very-big-is-coming-our-most-important-technology-project-yet/
[2] Putting the Wolfram Language (and Mathematica) on Every Raspberry Pi: http://blog.stephenwolfram.com/2013/11/putting-the-wolfram-language-and-mathematica-on-every-raspberry-pi/
[3] Mathematica Online: http://www.wolfram.com/mathematica-online/
[4] Wolfram Cloud: http://www.wolframcloud.com/
[5] WolframAlpha: https://www.wolframalpha.com/
[6] Wolfram: http://wolfram.com/
[7] About the Wolfram Connected Devices Project: http://blog.stephenwolfram.com/2014/01/launching-the-wolfram-connected-devices-project/
[8] Wolfram Connected Devices Project: http://devices.wolfram.com/