chi2Quant, FQuant and *Quant Functions

Discuss your code, get questions answered

chi2Quant, FQuant and *Quant Functions

Postby sokol_22 » Mon Jan 02, 2012 4:06 am

There are nQuant and tQuant functions in the Stat menu. How can one calculate quantiles from other distributions - chi2, F, .. ?
sokol_22
 
Posts: 9
Joined: Sat Oct 15, 2011 12:41 pm

Re: chi2Quant, FQuant and *Quant Functions

Postby oliver » Mon Jan 02, 2012 12:50 pm

Hi Jiri,

Sorry, right now, these are the only supported quantiles.

If you really need/want other quantiles, you could write the code for them, if you're so inclined.

Here's the code for the two supported quantiles, and URLs to were these came from:
Code: Select all
      "NormalQuantile": function(x) { // as per http://www.hpmuseum.org/cgi-sys/cgiwrap/hpmuseum/forum.cgi?read=182786#182786
         if (x <= 0) return +Infinity; else if (x >= 1) return -Infinity;
         var z, p = Math.min(x, 1-x), s = this.sign(0.5 - x);
         if (p < 0.2) { var u = -2*this.ln(p); z = Math.sqrt(-2*this.ln(p*Math.sqrt((u-1)*2*Math.PI))) + 1/(u*u); }
         else { var u = (0.5-p)*Math.sqrt(2*Math.PI); z = u + (u*u*u)/6;   }
         var k = 3;
         do {
            var t = (this.NormalDistribution(0, 1, z)-p)/this.NormalPDF(0, 1, z);
            z = z + t/(1 - t*z/2);
         }
         while (--k);
         return z*s;
      },

Code: Select all
      "StudentTQuantile": function(df, x) { // as per http://www.hpmuseum.org/cgi-sys/cgiwrap/hpmuseum/forum.cgi?read=183534#183534
         if (df != Math.floor(df)) throw Error("bad arg");
         if (x <= 0) return +Infinity; else if (x >= 1) return -Infinity; else if (x == 0.5) return 0;
         var t, p = Math.min(x, 1-x), s = this.sign(0.5 - x);
         if (-this.ln(p) < 1.7 * df) {
            if (p < 0.2) {
               var u = -2*this.ln(p);
               x = Math.sqrt(-2*this.ln(p*Math.sqrt((u-1)*2*Math.PI))) + 1/(u*u);
            }
            else {
               var u = (0.5-p)*Math.sqrt(2*Math.PI);
               x = u + (u*u*u)/6;
            }
            var x3 = x*x*x;
            var u = (x3 + x) / 4 / df;
            var v = (x3*x*x / 12 + x3 / 4) / (df*df);
            t = x + u + v;
         }
         else {
            var u = 2*p*df*Math.sqrt(Math.PI/(2*df - 1));
            t = Math.sqrt(df) / Math.pow(u,(1/df));
         }
         
         var d, k = 10;
         do {
            d = (this.StudentTDistribution(df, t) - p) / this.StudentTPDF(df, t);
            t += d;
         }
         while (d*d > 1e-30 && --k);
         
         return t*s;
      },


If you're interested to write code, let me know. I can give you a few pointers. The code above should be a good indication as to the effort involved.

Cheers.

Oliver

P.S. I'm waiting for Apple to release v1.4 (a big update), to make a big documentation update to the site. The quantile functions were previously undocumented. As soon as the documentation update is done, you'll find them in the Stats section of http://naivedesign.com/ND1/ND1_Reference__Functions.html.
oliver
Site Admin
 
Posts: 433
Joined: Sat May 01, 2010 2:11 pm

Re: chi2Quant, FQuant and *Quant Functions

Postby sokol_22 » Tue Jan 03, 2012 6:34 am

Hi Oliver,

The "hpmuseum" links seem to be broken. Anyway, I am interested in writing other quantiles functions. Your "pointers" will be appreciated.

Regards,
Jiri
sokol_22
 
Posts: 9
Joined: Sat Oct 15, 2011 12:41 pm

Re: chi2Quant, FQuant and *Quant Functions

Postby oliver » Tue Jan 03, 2012 6:14 pm

Hi Jiri,

Great!

hpmuseum links quickly become invalid, as posts become archived.
Here're the corresponding archive links:
http://www.hpmuseum.org/cgi-sys/cgiwrap/hpmuseum/archv020.cgi?read=183112
http://www.hpmuseum.org/cgi-sys/cgiwrap/hpmuseum/archv020.cgi?read=182786

I suggest you start there. I found this to be a very insightful discussion with great development from Dieter.

You probably want to start with the PDFs.
The PDFs, used by the two quantiles above, look like this:
Code: Select all
      "StudentTPDF": function(n, t) { if (n != Math.floor(n)) throw Error("bad arg"); return this.gamma((n+1)/2)/(Math.sqrt(n * Math.PI) * this.gamma(n/2) * Math.pow((1 + (t*t)/n), (n+1)/2)); },

Code: Select all
      "NormalPDF": function(mean, variance, x) { var a = x - mean; return Math.exp(-(a * a) / (2 * variance)) / (Math.sqrt(2 * Math.PI * variance)); },


These are small enough to develop on the device, I'd say. You just type "function(x) {" on the edit line, double-tap to expand it to multi-line, and type away (in JavaScript). There's plenty of JavaScript example code to look in the various downloadable shared folders.

Your quantile functions would likely be too large, to be written on the device. I'd suggest you implement the remote VISIT key mod, explained in this forum (a few threads down, and also mentioned, with link, in the hpmuseum discussion).

You do have FisherFDistribution(n1, n2, f) and ChiSquareDistribution(n, x) at your disposal. See the function reference for the meaning of the parameters. You can simply call them in your JavaScript code.

If you have C code (which would almost certainly be quite easy to find), it's quite easy to translate it to JavaScript. Sometimes all you have to do is change all type declarations to "var".

If you can develop the code on your computer in a JavaScript dev environment that might be easiest, though, with the remote visit key, it really doesn't take more than a second to try out code on the device, that you're editing on the computer.

Once you have a working user function, you can very easily make this into a system-wide function that works from everywhere (not only in one folder).
All you have to do is inject the code. You'll find examples of that in the various discussions about extensions, such as Constants, BigInt, or Color.
Manually, you can paste a line like this
Code: Select all
ME.FisherFQuantile = function(x) { ... };
into the JS Injection page (under Definition; after JS Injections are enabled in the Settings app), and that will make FisherFQuantile a command just like any other. (ME is new in 1.4 and stands for MorphEngine. It's an alias to calculator.functions mentioned in the JavaScript API documentation.)

If you get that far, maybe you want to share your work with other users? ND1 could use a Statistics folder, and your quantiles would be a great start to it.

Thanks, and good luck! I'm here, if you have questions.

Oliver

P.S. v1.4 finally posted.
oliver
Site Admin
 
Posts: 433
Joined: Sat May 01, 2010 2:11 pm

Re: chi2Quant, FQuant and *Quant Functions

Postby sokol_22 » Thu Jan 05, 2012 9:50 am

Hi Oliver,

I suppose that this.* function calls point to the ND1's inner functions.
Is it possible to test/run (directly) on my PC a javascript function which calls those this.* functions (without transfering it onto the ND1)? So the question might be if there is an ND1's emulator for a PC.

What is the best way to develop javascript functions on a PC? I use kind of an awkward method. I have an html file with a <script> block with the function definition. Then I call the function within the document.write command. For example:
Code: Select all
<html>
<body>

<script language="JavaScript">
function hello(){
return "Hello again!";
}
document.write(hello());
</script>

</body>
</html>

Is there a better way?

Jiri
sokol_22
 
Posts: 9
Joined: Sat Oct 15, 2011 12:41 pm

Re: chi2Quant, FQuant and *Quant Functions

Postby oliver » Thu Jan 05, 2012 1:50 pm

Hi Jiri,

A desktop version of ND1 is currently in development. It's called CalcPad (just like the upcoming iPad version of ND1). It will only be available in Q2, though. This will, as you suspect, make writing code quite a bit easier.

I just added a new program, getCodeAndRun, to the Tools folder. See the post in the other thread.
This should allow you to have your code in a text editor, click Save, and then push one key in ND1 to get your remote code (assuming IIS is set up fine, and you're saving into the directory IIS is serving) and run your code. The lag should be minimal, less than a second. Your code will run using the inputs from the stack, and deposit any output there.

I hope this makes for an acceptably comfortable environment. Let me know how it goes for you.

As far as writing and running JavaScript on a PC, I cannot recommend a tool beyond IE's quite excellent JScript debugger. You get to it by accessing the Developer Tools from a menu. I guess you still have to use a separate editor though.
You could also use Safari, which has also an excellent JavaScript debugger built in. That one (at least on the Mac) has an input line that's multi-line capable.

I'm sure there're many JavaScript development choices on PCs, but I do all my development on a Mac, so, sorry, but I can't give you better recommendations.

this refers to the current object. In the case of the functions we're talking about, this is the calculator.functions object, which contains all the functions that accept Reals. ME, again, is an alias to that object. So, instead of writing this.ln(), one could just as well write ME.ln(). As explained in the JS API doc, there're other function collection objects. For example, ME.complex holds all the complex number functions, ME.vector holds all the vector functions, etc. So, if you wanted the complex ln function, you'd say ME.complex.ln(). If you wanted the total function that sums up the components of a vector, you'd type ME.vector.total([1,2,3,4]), etc.
(Incidentally, while you're writing a function in ME.complex, you get ME.complex.ln if you say this.ln. That, really, is the reason for using this. It makes the code just a little more transportable between data types.)

For the quantiles, I guess you'll want to use the ChiSquareDistribution(n,x) and FisherFDistribution(n1,n2,f) functions. In a user function, you can just type these names without any namespace designation. MorphEngine does some things to blend the ME functions into a user function's namespace. You could also call them via ME.ChiSquareDistribution(n,x), and that's what you have to do later, if you want to blend them right into the ME object, via injection (ME.FisherFQuantile = function(x) { ... do something and use ME.ChiSquareDistribution(n,x) }).

Cheers.
oliver
Site Admin
 
Posts: 433
Joined: Sat May 01, 2010 2:11 pm

Re: chi2Quant, FQuant and *Quant Functions

Postby sokol_22 » Sun Jan 08, 2012 2:54 pm

I've assembled a prototype of the inverse ChiSquare function. I want to test it. I would calculate a list of reference results from let's say Matlab and then would compare it with the ND1's result list. How can one print a list of ~100 to 1000 values to the ND1's stack (perhaps one number per line)? I hope the stack can hold such a number of values (to be emailed).

Jiri
sokol_22
 
Posts: 9
Joined: Sat Oct 15, 2011 12:41 pm

Re: chi2Quant, FQuant and *Quant Functions

Postby oliver » Sun Jan 08, 2012 4:40 pm

Hi Jiri,

Congrats, that's great!

Producing a list of some thousand entries is no problem. Rather than dumping them on the stack, I'd recommend you produce an array, expand it, and email the stack.

Do you have a list of input test values?
If you have them on your computer, you can quite easily produce an array of them. See http://naivedesign.com/ND1/Project_Euler.html and search for "http". You can click on the links and see the kind of data these functions are importing. If you have a list that looks similar, you should be able to adapt the code snippets.

If you want to generate equally-spaced input data, use a program like one of these:

Code: Select all
≪ 1 1000 range 1000 / ≫


Code: Select all
≪ [] =arr
   1 1000 FOR i
      i 1000 / =arr[i]
   NEXT
  arr



Once you have an array of input values, run your function on it, like so
≪ [myFunc] MAP ≫
where myFunc is the name of your function in the current folder.

(You don't need to create programs. You can type this in just as well.)

This will give you an output array. Expand it by tapping the ...(xxx more) text. Then type email.

In the Tools folder is new runTests program. It will look for a variable named .tests in the folder you're in. That should contain an array of two-element vectors containing a command to evaluate and the desired result. This permits unit testing. For thousands of values, though, you're better off writing a little test program.

There's also a compare command. If you can import both your input data and your control data into ND1, you can run your code on the input data and then compare to the desired data by having both arrays on the stack and calling compare. It will report the indices in the array that do not match. So, an empty array as result means that all values matched.

Obviously, you can automate the get-input-data, run-function, compare-results steps all in one RPL program.
E.g.
Code: Select all
≪ getInputData
   1 1000 range 1000 /
   [ChiSquareQuantile] MAP
   getControlData
   compare
   DUP IF [] != THEN "Yikes! No match." ELSE DROP END



Hope this helps.

Oliver
oliver
Site Admin
 
Posts: 433
Joined: Sat May 01, 2010 2:11 pm

Re: chi2Quant, FQuant and *Quant Functions

Postby oliver » Mon Jan 09, 2012 3:28 am

If your function takes multiple arguments, you can still use MAP.
Use like so:
Code: Select all
array [2 3 myFunc] MAP

The array will then provide the first function arg, "2" would become the second, "3" the third.
If you wanted the array to provide the second argument, you'd say
Code: Select all
array [2 SWAP 3 myFunc] MAP

This would make "2" the first, array the second, "3" the third arg.
Finally,
Code: Select all
array [2 3 ROT myFunc] MAP

would make the array value the 3rd arg, and "2" and "3", the first and second arg, respectively.

If you later inject your function (something like ME.ChiSquaredQuantile = ME["Statistics"]["myFunc"];), the calc will automatically run your function over all elements of an array, when you call your function on an array. That is, you won't need MAP (unless you do have multiple args).
That works, because you choosing to inject the function into ME (vs. ME.vector, or ME.complex, say) makes the calc know that your function expects a real value.

As a user function, the calc doesn't do that, because it doesn't know what data type your function is expecting. For all it knows, your function may be defined on an array, in which case you must be passed it, instead of individual elements. (There's a plan to add the option for typing a user function's input args, something like function (x:real), and that will enable the calc to know what type it's expecting.)
oliver
Site Admin
 
Posts: 433
Joined: Sat May 01, 2010 2:11 pm


Return to Programming

Who is online

Users browsing this forum: No registered users and 1 guest

cron