Primitive root mod p

Transire suum pectus mundoque potiri

Functional Programming in Python 26 April, 2008

Filed under: Programming — Nikolas Karalis @ 2:37 pm

Recently, I decided to learn how to use the functional programming tools provided by Python.

Truth is, it’s interestingly strong.

The whole concept consists of using 3 built-in functions (map, filter, reduce), lambda functions and of course the beloved List Comprehensions.

In order of appearance :

map (function, sequence) : It applies function on every item of the sequence.

You can even provide a function which takes two arguments and use it like this :

map (function, seq1, seq2)

filter (function, sequence) : It returns a sequence of the items which have a function (Item) == True value.

reduce (function, sequence) : It applies the function (2 arguments) to the first two items of the sequence, and then to the result and the third etc…

List comprehensions :

This a powerful feature of Python.

I will explain it by an example :

>>> a=range(10)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

>>> [x**3 for x in a]
[0, 1, 8, 27, 64, 125, 216, 343, 512, 729]
Lambda functions :

With lambda functions you can create anonymous functions on the fly.

For example, the normal definition of a function is :

def f(n): return n**3

With lambda functions :

f = lambda n: n**3

In both cases, we use like this :

>>> f(5)
125
An example of using lambda functions with the map function would be like this :

>>> a=range(10)

>>> map(lambda x: x**3, a)
[0, 1, 8, 27, 64, 125, 216, 343, 512, 729

This is exactly the same as the example in list comprehensions.

So, with all the above we have a whole new range of tools for dealing with lists.

 

Seismology, Snow & Procrastination 18 February, 2008

Filed under: /dev/random, Programming — Nikolas Karalis @ 1:29 am

I haven’t updated my blog for a while. No specific reason… I was just procrastinating since exams were coming and I didn’t have much to say.

The latest news from Greece are the earthquakes and the snow (and of course my exams :-P ).

The last few days, 2 big earthquakes (6.2 R and 6.9 R) shaked Southern Greece and last night, it began snowing in Athens!

This happens once every one or two years, so it is big news

So last night, a small earthquake reminded me of the passionate discussions about the earthquakes and since i didn’t have much to do because of the snow and my constant last month procrastination, i decided to build a database of the Greek Earthquakes of the last few years.

So, a little bit of coding and a few hours later, I proudly (:P) present my Greek Seismological Database (click here to visit).

EDIT : A new addition : It is automatically updated when a new earthquake happens…

I end this post, with a few photos from Athens in white.

*******

*******

 

Thesis Database and Python CGI uploading. 28 January, 2008

Filed under: Programming — Nikolas Karalis @ 6:39 pm

Before a few days I had the idea that it would be really nice if we could have a database of greek theses and dissertations, about mathematics and science in general. From what i know, there are a few databases around, mostly for Electrical Engineering and Computer Science dissertations. So, I thought that it would be a good opportunity for me to exercise my CGI and Python Web scripting skills.

And here I am, presenting the Thesis Database project. I hope that it will be useful and people will contribute.

But while coding the CGI backbone, I had a few problems to solve, so since i had to come up with the solutions (couldn’t find anything useful online), i decided to post them here, for future reference. I will also give the basic idea of how a python cgi uploading script works. The focus is on security of the code.

So, the following is a very simple html form, which will be used as the user interface for the upload.

We suppose that the cgi script is called upload.py and is placed inside the $Web root$/cgi-bin/ directory.

upload.html

<html>

<head> <title>Upload Example</title> </head>

<body> <div align=”center”>

<form action=”/cgi-bin/upload.py” method=”POST” enctype=”multipart/form-data”>

File : <input name=”file” type=”file” size=”35″><BR>
<P><input name=”submit” type=”submit” value=”Upload”>

</div> </body> </html>

(more…)

 

Introduction to Algorithms (Greek Translation) 13 January, 2008

Filed under: Computers — Nikolas Karalis @ 7:09 pm

The following post is written in Greek. It is my complains about the Greek translation of the classic book : “Introduction to Algorithms“, T. Cormen et al.

Αυτές τις μέρες και με αφορμή κάποιες ασκήσεις Αλγορίθμων και γραφημάτων, σκέφτηκα να εκμεταλευτώ την ελληνική μετάφραση του κλασικού βιβλίου Αλγορίθμων των Cormen, Leiserson, Rivest, Stein απο MIT Press, που εκδόθηκε πέρυσι στην Ελλάδα απο τις Πανεπιστημιακές Εκδόσεις Κρήτης (ΠΕΚ).

Οι ΠΕΚ είναι κατα τη γνώμη μου απο τις καλύτερες εκδόσεις στην Ελλάδα, με πλήθος αξιόλογων βιβλίων. Και είχα χαρεί πολύ για την μετάφραση αυτού του πολύ καλού βιβλίου. Αλλά πραγματικά, είναι μια απο τις χειρότερες μεταφράσεις βιβλίων που έχω διαβάσει ποτέ.

Όπως αναφέρεται στον πρόλογο του επιμελήτη :

“Μία απο τις κύριες μέριμνες των ΠΕΚ είναι η προσεκτική χρήση της ελληνικής γλώσσας… Ένα απο τα σημαντικά προβλήματα που έπρεπε να επιλυθούν στη συγκεκριμένη έκδοση ήταν η εύστοχη απόδοση στα ελληνικά της σχετικής επιστημονικής ορολογίας, και η συνεπής τήρηση των όποιων ορολογικών επιλογών καθ’ όλη την υπο έκδοση σειρά.”

Και τα κατάφεραν. Μετέφρασαν κάθε αγγλική λέξη στα ελληνικά, και μάλιστα σε λέξεις που προσωπικά δεν έχω ξανακούσει να χρησιμοποιούνται. Και γίνεται σε τέτοια έκταση που μοιάζει με αυτόματη μετάφραση του Google. Φυσικά δεν κρίνω την ορθή χρήση

των ελληνικών απο τον μεταφραστή αλλά την υπερβολική και (κατά τη γνώμη μου) ενοχλητική και λανθασμένη μετάφραση των επιστημονικών όρων. Πραγματικά λυπάμαι τον άνθρωπο που θα δοκιμάσει να χρησιμοποιήσει αυτό το βιβλίο για να μάθει αλγόριθμους. Προσωπική μου εκτίμηση είναι οτι καλύτερα να αγοράσει κάποιος την αγγλική έκδοση (η οποία κοστιζει 55 ευρώ) παρά την ελληνική (της οποίας ο πρώτος τόμος μόνο κοστίζει 40).

Δεν θα έμπαινα στον κόπο να γράψω για αυτό το θέμα, αλλά κάθε φορά που ανοίγω αυτό το βιβλίο, με πιάνει πονοκέφαλος απο την ακατανόητη μετάφραση.

 

Python Hacking 7 January, 2008

Filed under: Programming — Nikolas Karalis @ 5:43 am

Efficiency Tips

In this post, i will try to collect tips and tricks for efficient Python programming. Feel free to contribute.

I will update every once in a while, so check back…

  • Fastest string conversion :
>>> a=123
>>> `a` 
'123'
  • Fastest string concatenation :
>>> ''.join['a','b','c'] 

Efficiency comparison

Optimization Anecdote (Guido van Rossum)

  • Generator Functions
>>> a=[1,2,3]
>>> b=[x**2 for x in a] 
b=[1,4,9]
  • Timing a process :
from time import clock
t=clock()
... 
t=clock()-t

(more…)

 

Deleting duplicate lines from file 5 January, 2008

Filed under: Programming — Nikolas Karalis @ 10:27 pm

I’ve been fighting with a computational problem for many days, but i haven’t even come close to an acceptable solution.

The problem statement :

Given a file of n lines, return the index number of the duplicate lines (where index line is 0 for the first line, 1 for the 2nd etc.)

It may sound trivial, even silly, but i can assure you it is not. It can be relatively simple for small number of lines.

But as n raises, the problem becomes exponentially harder.

When i first faced the problem, i had to deal with a few hundred thousands of lines. So, i came up with a simple python code to do it.

When the first difficulties appeared, i came up with a faster solution.

  def duplicates(sequence):
	visited={}
	dupl=[]
	for x in sequence:
		if x in visited: dupl.append(x)
		visited[x]=1
	return dupl

However, now that i have to find the duplicate lines in files with 15 million. 26 million and more, it is impossible to use this code, since it returns memory errors.

So i found another idea, which is REALLY slow for now.

  1. Sort the file with the windows or unix command : sort
  2. Use unix command uniq -d, to get a list of the duplicated lines.
  3. Use unix command grep -n, on the unsorted file for every line in the previous list, to get which lines are duplicated.
  4. Use a simple python script to parse the result and get only the the integers we are interested in.

However, the grep part is REALLY slow for huge files. So, my problem remains. However, i reduced the problem from removing duplicated from a file, to simply getting fast the index of a line in a file or equivalently, fast iteration and comparison of the file lines.

After extended digging in the Internet, i was not able to find any efficient algorithm or implementation.

 

OpenBSD & Security 5 January, 2008

Filed under: Computers — Nikolas Karalis @ 9:36 pm

I’ve spent the last few days setting up a new OpenBSD server, focusing on the security aspects.

I’m not a security expert, but during this days I’ve learned a few things which i would like to share.

So, I will describe these security tips, without getting into much detail, from general to special. For more details on these, just google or ask me.

Firstly, why OpenBSD?

Well, i had no experience with this OS, so i thought to give it a try. Furthermore, OpenBSD is (generally?) considered to be (one of) the most secure systems around. So, why not?

1) When connecting remotely to your server, always do that with SSH2.

2) You should not login over SSH as root. Create an account, add it to the wheel group so that you can
su with it, and change the PermitRootLogin option to no in /etc/ssh/ssh_config .

3) Use long passwords. You can even set the min. password length as you wish.

4) Always use the latest release, and always patch as soon as possible your system, when a patch is available.

5) Enable only the needed services, and check thoroughly for open ports. e.g. Disable the inetd and sendmail f you don’t need them.

6) After you finish your configurations, change your security level to 2. This is the highest possible, but be aware that this can make administration more difficult.

7) Change your web server’s directives, so that you don’t show to people which server you use.

8 ) Enable your swap partition encryption (this is enabled by default).

9) Use pf (packet filter) to create strict rules for your firewall. For example, block all incoming traffic, and only enable the few services that you really need. In addition to that, use antispoof and packet normalization (scrub) rules.

10) In your pf rules, include rules against bruteforcing. With this, you can avoid DoS (Denial of Service) attacks.

11) Block incoming traffic based on ip, for example allow ssh logins only from your local network or what fits your needs.

12) Try to fool nmap (using pf), so that you not reveal which of your ports are open, and which OS you are using and also log the scanning attempts.

13) Change the sysctl.conf flags, so that your system reboots automatically during a kernel panic.

14) Use file flags like schg, sappnd and uappnd to disable deletion and editing of specific files and directories, like /var/log.

15) Keep a track of your files’ checksums, and your downloads’ checksums.

16) Use snort (or other NIDS and Host IDS) in addition to the pf and system logging to prevent and identify security breaches and attempted attacks.

17) You can use honeyd to create virtual hosts and honey traps for the script kiddies.

18) Use SaDoor for remote stealth administration.

19) If possible (if you are administering the router used), use static ARP in your network, to avoid ARP poisoning attacks.

 

Facebook! 8 September, 2007

Filed under: Computers — Nikolas Karalis @ 7:40 pm

The last few months, facebook is open for everyone and not only university students.

I am generally opposed to these websites, but Facebook is one of a kind.
It is aesthetically nice, easy to use and extremely popular (at least in North America).

I tried it, and now i’m using it all the time.

I found so many friends of mine from around the world, that we hadn’t talked in so many years…

So, Facebook me!

Here, is a link to an interesting article regarding Facebook :

Facebook’s plan to hook up the world

 

Learn Genetics 13 June, 2007

Filed under: Computers — Nikolas Karalis @ 2:03 am

I just found a great website… It is hosted at University of Utah.

It has many entertaining and educative Flash presentations of how Genetics works.

http://learn.genetics.utah.edu/

Special thanks to primelude for sending me one of these files… ;)