UMIT

Umit

the nmap frontend

Summer of Code


Google Summer of Code - Proposal ideas - 2010



Umit is participating in GSoC 2010 through Nmap. Send your proposal!
Check also Nmap Ideas

Send your proposal: GSoC - Google Open Source Program

Umit Network Monitoring Suite

After 5 years of Umit, we now have a respectable range of network monitoring tools: the Umit Network Scanner (formerly known only as Umit), Umit Packet Manipulator, UMPA, Umit Bluetooth Scanning and Sniffing, Quick Scan, Network Inventory, Zion, Audit Framework, NSE Facilitator and Umit Network Scanner Web Interface (formerly known as UmitWeb). Now it is time to integrate everything into the Umit Network Monitoring Suite, and we want a Google Summer of Code Student to get it done this year. The goal here, is not to bundle everything into one executable software, but to have them to be shipped together and allow their proper interaction.

Find bellow a quick list of features to be develop to form this suite:

  • Ensure that all projects are following the same usability standards
  • Define sane standards to be followed by all apps inside the suite
  • Create an installer capable of installing all apps, and allow user to select which should our shouldn't be installed
  • Make apps aware of each other, developing features that would allow one to call another based on it's current context. For example: Umit Network Scanner could launch the Network Inventory to check on data related to a given host that was just scanned.

Network Scanner Improvements

This is our oldest and most solid tool, but we can't lay down and expect it to reign forever. Network Scanner needs to evolve and follow up on our users needs, improving continually it's performance, usability, portability and reliability. The perfect student for this slot would easily identify spots where Umit it can be improved, and describe it in a consistent proposal and how it's proposals would impact in our user's experience. New features are very welcome, mainly if they come to improve usability and integration. We need improvements in the search window, database, scan comparison window, reports generation, etc. Following examples:

  • UmitDB-NG was created to support Network Inventory. But Network Scanner have another old database containing all XML. Actually we're using XML stored on database and other database with each field stored. The idea is merge both and create a good approach for this problem. In this process UmitDB-NG should be updated, creating a layer to support more than one backend: ZION, Nmap, etc.
  • Improve scheduling in Network Invetory (Cronjob syntax is not easier for casual user)
  • Fix bugs founds on Trac (Integration, etc) - See open tickets of Umit in [1]
  • Merge Preferences Window
  • Merge NSE Facilitator

Zion Integration

In the GSoC (2009) we start to develop a tool to improve the OS fingerprinting classification performance. The tool named Zion is almost finished as a back-end and need to be integrated to Umit Network Scanner. There are two main documents related to current development status [0] and another one to help you understand the theoretical background of Zion [1]. The three main steps to do this project is:

  1. Adapt Umit Network Scanner to support Zion (and possible other backends);
  2. Finish Zion front-end and integrate the back-end to Umit Network Scanner; and
  3. Test Zion performance across a considerable number of operating systems and network configurations (real and virtual).

References:

Umit Bluetooth Scanning and Sniffing Integration and Improvements

Though our bluetooth projects are well functioning, we need to get them improved with bug fixes, increased compatibility, improved usability and integration to our other network monitoring tools.

The Bluetooth Sniffer[0] project presents itself a general framework to decompose Bluetooth protocol data units and perform a PIN crack attack through a module. However the project needs to get up to speed on bug fixes and integration into Umit Packet Manipulator[1]. The idea is to strengthen the code base before embarking on added attack modules which would further shape and define the project.



PacketManipulator - Improvements

PacketManipulator is a extensible network scanning frontend. The main features is forge custom packets and send them over the network, sniff traffic, manipulate packets, etc. Umit Project have been working on improving and add enhancements in this project all the time and we get far with it. Towards to keep high expectations of our users we are proposing bring improvements to PacketManipulator. See in line:

  • Create filtering mode (a better filter metadata-based and keywords) - filtering by MAC(dst, orig), IP, Port, protocol etc, with expression system supporting ranges in all fields, similar Wireshark.
  • Improve drag-drop Packets: create an automatic-way to fill all stack automatically (but it should optional) - manual mode should works as well.
  • Actually there is features just working with scapy patched under deps/scapy-patches, but PacketManipulator should can run without any patches. Just if features are available it should use.
  • Finish my branch to integrate UMPA ( I already started but it's not finished yet)
  • Improve PlotStatus plugin and integrate it (Just a small plugin that made with statistics)
  • Improve Message Sequence Flowchart ( Plugin by Kasina) and integrate it [2].
  • Create more decoders - passive audits [1] (Like HTTP/SMB decoder developed last GSoC), ie - IRC, Jabber, VoIP (SIP, RTP, etc), SNMP, SNMP, IMAP, etc.
  • Customize colors in sniffed packets
  • Test it on all platforms and fix bugs (ie, Windows)
  • Other independent-features proposed by your own

UMPA link-layer support

UMPA [0] is a library to manipulate packets over network. Basically, user can modify packets on each layer of OSI model. In current state of projet, link-layer (second layer) is not supported. An eperimental branch[1] adds this functionality but it still needs some improvements and testing. Porting to Windows and MacOSX is essential (in current state, only Linux is supported). Link-layer support should be also fully covered by unit-tests to avoid occasional bugs. In the end, the branch should be merged to main branch.

References

Remote Network Monitoring or Network Inventory with Super Powers

With rising of computers and networks, there was a significant increase in number of devices to be management. Network Inventory is a nice tool for keeping track of hosts in your network throughout the time, but it doesn't give you much more than that currently. Our goal is to give it super powers this year, by allowing it to receive SNMP traps from hosts and keep track of their status. Besides the capability of receiving SNMP traps, we would like to create a protocol for keeping track of hosts and services status in a client/server manner, where hosts would report their status to the Network Inventory through small monitors installed in the host.

Students should keep in mind that Network Inventory is then expected to turn into a server, and that it should be able to support failing back to a slave server and all monitors should be aware of that also.

We expect Network Inventory to integrate with Umit Mapper, in order to provide usable graphs about the overall network status and points of failure as well as support integration with Umit Plugins to allow our users to adapt both the Network Inventory and the monitors to their specific needs. Allow easy development of cross platform monitors, thinking that our users may want to monitor *ANY* kind of networking device, services and files. User should also be able to define alarms, and set thresholds that would trigger them.

It certainly isn't an easy project, but it is a very exciting one, and students will be facing several challenges that will certainly boost their resume and strengthen their experience and gain a sustainable visibility on the Internet through this project. A suggestion for this proposal: 3 software components to implement:

  • Agent: It can be a sniffer to monitoring events configured by user remotely (Example when HTTP arrive, when too many traffic is going around in the network etc) - It can use UMPA library.
  • Broker: It receive SNMP messages and agent events (using ad-hoc protocol). It store all events in database. It will be available to access all stuff by the webservices. It is a kind of gateway between monitoring and agents.
  • Remote Monitor (Viewer): view all events and details (alerts etc) - it can use Network Inventory to list alerts, traffic etc. It will be able to configure the Agents and send SNMP messages,



Usecase to help understanding idea, see figure.


NSE Facilitator

NSE (Nmap Script Engine) is a very extensible Nmap feature enabling users to write your own scripts to customize your network scanners. [1] We have been working on this feature, and we get far with it. But still, we have a lot of work to do. The goal of NSE Facilitator is to provide a good NSE experience for average users, who doesn't know NSE much but still need to use it to get their job done. And our work, is to help them have their job done quickly. We need some ideas on how to improve NSE, and ease its use for the average user. Students willing to apply for this project, are expected to interact with the plugins project, as they're closely related projects and one can benefit or injure the other project. If you're willing to work on a proposal for this project, take a look at where we currently are with NSE Facilitator at https://svn.umitproject.org/svnroot/umit/branch/NSEFacilitator .

  • [1] - Nmap Network Scanning (Chap. 9 - Nmap Script Engine) or a preview (http://nmap.org/book/nse.html)

Umit Web Scanner

The Umit Web Scanner goal is to provide a tool for assessing Web applications, and checkup on their known vulnerabilities, compliance with standards, benchmarking and comparing all results with sane patterns of what a regular web application should score. This Web Scanner would also suggest possible improvements to user, and allow him to develop it's own plugins to test specific characteristics of it's web application.
This project is intended to be integrated into Umit Network Scanner, rather than being an independent application. This will leave the student with extra time on developing more features for the project than if he was expected to create a whole application from scratch, and will help us keep our software perfectly integrated.

Using AI for OS fingerprinting

There is huge number of tools and techniques that perform OS fingerprinting. Each technique has your own strengths and weakness. Some of them already have an Artificial Intelligence foundation, some are just pattern matching rules. The idea behind this proposal is to combine these techniques into a hybrid intelligent system that take advantage of each technique. For example, we can build an expert system from the information the is avalible about some OS network behavior (e.g. IP TTL, TCP Options sequence, and so on), and accomplish the result of this expert system to the result of Zion that uses a more complex approach (neural network and pattern classification).

To perform this work the student must be familiar with AI and network OS fingerprinting, otherwise he/she must spent some time before GSoC studying this matter. See the references for a good start.

References:

Nmap OS fingerprint database system

The Nmap OS fingerprint database is a set of signatures which represents many TCP/IP stack implementations of operating systems. How this system works is not easy to understand, but Nmap always has a good documentation of its features. In this case the student that wish apply this idea must be familiar with the Nmap OS detect documentation [0].

This idea consists on a solution for two main problems:

  1. Nmap OS fingerprint system fails in some cases [1], but this is not because the database data, this happen because the design of the fingerprint matching algorithm. So, using a selective OS matching algorithm with the nmap-os-db file it's possible solve these problems without changes in Nmap code base. The figure below show an example of Nmap fingerprint on his database.
  1. Nmap results of its OS fingerprint system only can analysed from user statically. It'll interesting if users can choose the signature fields and algorithm to use when perform OS matching. The figure below illustrates the whole process of OS fingerprint.

The choose of the OS matching algorithm (component 5 in the above figure) depends on the format of the input, in this case a signature from nmap-os-db file. If you want to convert this alphanumeric data into numeric values you will amplify the possibilities of OS matching algorithm that can be used.

References

Vulnerabilities database system

The Umit classification for vulnerability score of hosts is based only on the number of ports that Nmap found. This can be improved using each port information like its service and version. To do this task is interesting create a relational database SQLite [0] compatible and an API to easily access database information (e.g. functions that return a vulnerabilities set give an service and version). Not just the services can be search in database but the operating system detected by Nmap too. Good database candidates are the National Vulnerability Database [1] and The Open Source Vulnerability Database[2].

Beyond the database application interface would be good if the tool proposed has a user interface that give to the user the option of search for vulnerabilities and view them with a friendly form.

Look around some vulnerability search engines to see what they have, and what they don't have.

There are something in development. Please check: http://trac.umitproject.org/browser/nvdb

References

UMPA extra improvements

There are several features which would be nice to implement in UMPA [0].

  • new protocols implementations (e.g. ICMP and other supported by PacketManipulator[1])
  • finish auto-generation for some fields in already implemented protocols as IP, TCP
  • rewrite XML extension to support SOAP instead of DOM
  • IPv6 support
  • close tickets related to UMPA[2]
  • own suggestions!

References

PacketManipulator, a distributed approach using Audit Framework

Nowadays home-networks, enterprise and others is growing fast using high level equipment's: switching, routers and other moderns approach in operators world. For academic and industrial world the sniffing mode is used all the time. Unfortunately it's not easier to access to all the traffic in the networks because equipment don't flood traffic like old routers, but it could be done communication between machines in Local Area Network (LAN) - see figure.

A first approach is sending sniffed traffic (in each node) through the network to a centralized server. Unfortunately it produces other traffic to transfer so this extra traffic should be removed in final result. Another task that should be done is synchronize the clock using NTP protocol [1].

1. The first step of project is adding feature of PacketManipulator running in background ( no GUI allowed ) and loading audit extensions.
2. Developing an audit extension to exchange traffic between nodes in the network ( it just send the traffic to the monitor sniffer - centralized architecture)
3. Write a wrapper to receive traffic from other nodes
4. Connect with wrapper and showing the packets in Sniffing Prespective.
5. See all traffic in Local Area Network (LAN) - synchronize time.

    [1] - http://www.ntp.org

    Quick Scan Improvements

    Quick Scan's main goal is to provide our users with the same searching experience users have while using Mac's Spotlight or Google Desktop's Quick Find and others of the like, where results keep appearing on the screen as user types in the search query.

    Last year we've gone pretty far with Quick Scan development, but we need to go further. We consider Quick Scan as a very important application inside our suite, and it provides our users with a very high usability experience, and is a very powerful, easy and quick resource of information about the network without requiring the user to launch a big app (like Umit Network Scanner) or user the console (having to remember nmap command options).

    Students willing to propose for this idea, should check out the quick scan from our repository repository (http://trac.umitproject.org/browser/branch/QuickScan), and give it a try before writing their proposals. There is already a lot done for this project, but we need more. Be creative and audacious.

    UmitMapper new features

    Idea it's add new features to Topology Network as called UmitMapper. Some topics that can be explored:

    1. How represent graphically the existence of services in hosts.
    2. Which others visualization techniques can be used to make visualization better.
    3. What more kind of information can be expressed by the map.

    Look around for other network visualization tools[1,2] and see what they have, and what they don't have.

    References

    Umit Network Scanner Web Interface

    Formerly known as !UmitWeb, the Umit Network Scanner Web Interface is the web companion of Umit Network Scanner, and it's main goal is to allow network administrators to scan their network remotely without having to go through a VPN or SSH, but simply by using any regular web browser. Although being a good tool, it is becoming outdated, and we need something with a good experience with web development get it back on tracks using the most recent resources for a good usability experience.

    We are open to any good idea, from using pure Java Script to Flex. Also, think about developing web components that would provide realtime stats of the network or provide any information that the user may find relevant to be deployed in any regular website. For example, a Network Administrator could put a web component in his blog with stats like number of hosts, number of closed ports, vulnerability level, etc. from the network he manages or another network.

    That would be interesting, mainly if he wants to prove how good he has being doing at work ;-) We're sure you'll find other uses and other useful data that could be displayed in those web components also.

    Umit Assistant

    Clippit Everybody knows Clippit, the animated assistant that features Microsoft Office. It would be nice to have one a animated assistant to take care of helping users in some situations, and I'm sure that it is going to help Umit on every new feature that it might have in the future, showing to the user how to better analyse the scan results that he got, suggesting things that he can do with the result, and even what that result means. There are a lot of things that can be done with it. The student willing to apply for this idea, must take care of making it extensible in a manner that makes it easy to be used and integrated to future projects. Also, it is a good idea to support assistant switch, letting users switch the assistent from (for example) a animated pill to a animated HUB (??). Volunteers for the animations and designs are very welcome to help our studnet on this task! And, of course, we're going to give you the proper credits for your work on our website and at the Credits window of Umit.

    Nmap Wrapper for Python

    The wrapper intention is to provide a module from which you can create an Nmap instance, set the desired options and targets and run it without the need of executing it in another process. The wrapper must allow access of Nmap funcionalities like estimated time to finish the scan and runtime user interaction.

    User's almost won't note any change with this wrapper. This is a feature that is going to make Nmap and Umit developers life's easier, while adding new features or even integrating with future Nmap options and functionalities.

    Independent Features

    This is the spot of those willing to face whatever it comes for Umit: bug hunting, feature development, installation scripts, portability improvements, usability improvements, new small and medium size features, etc.

    Your Great Idea!

    We truly believe there is a lot of creative and talented students out there with exciting ideas waiting for a chance of a GSoC or USoC to be developed, and we're wide open to receive them! We want to hear from you, whatever your idea may be.

    Don't feel ashamed, don't feel restrained by our own ideas, and give your great idea a try with us. Umit begin as a dream also, and we think that your dream could as much or even more successful than Umit currently is.