Planethunters: How to download Kepler lightcurve data! (for basic users)

As promised, here’s the second part of my tutorial (following on from my previous post), explaining how to download Kepler lightcurve data for the stars on planethunters.org! Note however that this is intended for somewhat more advanced users than the previous one – you’ll need to know your way around Microsoft Excel for this!

EDIT: So far (as of the 4th Feb 2011) the nsted site doesn’t have the Q2 data yet. If you want to download the original Q2 data, or you’re looking for the original FITS data, then please refer to my more recent post describing how to download the FITS data from the MAST site!

Alternate Search Method

But first, an alternate search method! dale_j brought this to my attention on the PH forum – there’s another site that you can use to track down stars and download the lightcurves from, and it’s located at http://nsted.ipac.caltech.edu/applications/ETSS/Kepler_index.html.

This Caltech Search form is a bit easier to use than the Kepler Search form in the sense that you can just directly enter the minimum and maximum values that you’re looking for in the relevant text boxes. So instead of having to setup the Radius box from the User-defined Fields on the Kepler Search site and typing (for example) “0.55 .. 0.65”, on the Caltech Search form you can just scroll down the page to the Radius box at the bottom and type 0.55 in the Min Value box and 0.65 in the Max Value box there. Enter the values for Visual Magnitude and T_eff as well (you just need to enter the same value for T_eff into both the Min Value and Max Value boxes) and click the Submit button, that’ll give you the results! However, the Caltech Search doesn’t do allow you to specify which Quarter to search in, so you’ll get results listed from both Q0 and Q1 – in the results you see a Start Time and End Time column, and the Q1 file is the longer one (about 34 days in duration).

The other handy thing about the Caltech site is that it allows us to easily search for stars for which there is no data, which are the stars listed with “Unknown” Teff/Radius on Planethunters. To do this, enter the visual magnitude range, and tick the checkbox on the Teff and Radius rows that says “include stars with no value”, and hit “Submit”. This brings up a long list of stars, but they can then be manually searched to find the lightcurve of the star in question (if you’re so inclined!).

I was going to explain how to download the data from the Kepler Search site I described in my my previous post, but actually it’s easier to do from the Caltech site.

If we type in the parameters from the previous post (Teff = 6792, KEP Mag = 13.25 to 13.35, Radius = 1.55 to 1.65) on the Caltech site, we again get a KID number (shown as “Star ID” here) of 11092273, which is a good start!

Viewing the lightcurve

Once you’ve got a result, you’ll see that there are two links in the “Star ID” column – “Plot Time Series” and “Compute Periodogram”. Click on the “Plot Time Series” link for the Q1 data (don’t worry about the periodogram for now) and you’ll get a graph showing the lightcurve. The default is to plot the graph with red dots marking the datapoints and with no line between them, but you can change that.

On the right of the graph you’ll see a box with Range, Line, and Symbol sections. You can use the pulldown menus in the Line and Symbol sections to change the appearance – for best results, I suggest that you add a Solid line in the Line section, and turn the x to Dots in the Symbol section (note that if you change the Symbol to “none” but also have Line set to “none”, you won’t see anything on the graph!). Hit the “Update Plot” button and that will redraw the graph. Drawing the Line is very handy since it’ll reveal the true pattern of the data (that can be obscured if there’s a shutter effect on the lightcurve).

Downloading the data

This is all very well, but you want to download the data and do something with it yourself, right? Under the “Update Plot” button you’ll see two links “Download Time Series” and “Convert Table” – click the “Convert Table” link. A new window will then appear asking you to select a delimiter from a dropdown menu – this is the symbol taht the columns of the data table are separated by (a bar, semicolon, or a comma). Select “comma”, and press “Convert Table”. Nothing much will happen, but you can now click the link below it that says “Please click to download your converted table”.

The table is saved in csv format (comma-separated variable), and Excel can open it up straight away if you left-click it. However, I strongly recommend that you right-click the link and select “Save As..”, which will allow you to save the file in a location of your choosing. If you left-click it then it will open up directly in Excel but you must save it elsewhere from there, otherwise it just opens in a temp location and will be lost when you close it!. Once you’ve downloaded it, double-click the file to open it in Excel.

Plotting the graph in Excel

Once you’ve got the data, you can do whatever you like with it! Here I’ll explain how to draw your own lightcurve plot – at this point, I’ll assume that you have some expertise with spreadsheets (otherwise you probably wouldn’t be downloading the data in the first place!). I’m assuming you have Excel 2007 here, so you may have to adjust what I say if you have a different program, but the principles should be the same.

1) Once the file is open in Excel, you’ll see a rather huge header section taking up the first 288 or so lines. This is kinda handy if you want to know what all the columns are (below that), but not useful for graph drawing. If you have Excel, then just select all the numbers below the header (including the title rows at the top) and cut and paste them into the top of the graph, or into a new sheet. That way you should have the MJD column in column A, with the first title cell in the row 1.

2) Next, you need to remove all the cell values of “Null” so that the graph we plot won’t get screwed up by those – they’ll be replaced with blank cells, which won’t plot as anything at all. Bring up the Find/Replace box (in Excel, hit Ctrl+H), type Null into the “Find what” box, make sure the “Replace With” box is completely empty, and hit the Replace All button. There should be about 13,000 replacements made.

3) Now you need to expand all the title cells so that you can see the full text. There’s a little diagonal arrow in the top-left corner corner of the sheet, between the A column and the 1 row. Click that and select the whole graph. Then move your mouse over one of the dividing lines between the columns (e.g. between A and B) until it turns into a cursor with arrows pointing left and right, and doubleclick the left mouse button at that point – this will change the width of every cell in the table to fully include the title rows so you can see what’s in them.

4) The columns we want to plot in our graph are the MJD (days) and AP_CORR_FLUX column (Columns A and J). The easiest way to do this in Excel is to select columns B to I and delete them completely (right-click them, and click Delete) – this should shift the contents of Column J (if that’s the AP_CORR_FLUX column) to column B (right next to the MJD days in column A).

5) Now select Columns A and B, and go to the Insert tab on the ribbon at the top. Click Scatter in the Charts section, and select the second graph type in the top right of the dropdown (Scatter with Smooth Lines and Markers). You should see a small graph pop up, showing a blue mess of dots – this is because (a) the graph is too small, and (b) the dots are too large.

6) Click on the graph and expand it to fill more of the screen by dragging the arrow handles in the corners of the graph – that should make it easier to see. You can also delete the graph legend on the side to make more room on the sheet. Then rightclick the data points and you should see a menu box show up – click where it says “Format Data Series” at the bottom. A new window should pop up that (among other things) has “Marker Options” and “Line Style” on the left of it. Click on “Marker Options”, then click “Built-In” and change the size to “2” – this reduces the size of the datapoints. Alternatively you can select “None” for the Marker Options, which removes the dots completely – but I find them to be useful to retain because then you can see exactly what the data is doing. Then click on Line Style and change the width to “1 pt” (you may get a message box saying that it could take time to draw the points, but just press OK on that), and then Close the window.

With that done, you should have your graph – now you can zoom into any area you like or change the y-axis scale by adjusting the ranges on the graph axes (or do anything else you want with the data). This is what the graph for APH10135234 looks like in Excel (MJD Days are the x-axis, AP_CORR_FLUX is the y-axis):

If you want to be more fancy about it, you can edit the data so that the MJD are measured in days from 0 to 34 by subtracting the lowest integer value of MJD from all the MJD cells (e.g. if the MJD values start at 54964.51105 then subtract 54964 from all the values in that column), and you can normalise the flux by dividing all the Flux values by a number in the middle of the vertical range (e.g. if the mid-line of the flux is about 109350000, then divide all the Flux values by that number to get it centred around 1).

(I haven’t touched on the other stuff on the Caltech site – I’m still figuring that out! The periodogram looks particularly interesting though, as it seems to help find perodic patterns in the data!).