Register now and start sharing your code snippets.

How to parse RI generated documentation using RDoc and Ruby

Ruby posted 11 months ago by christian

RI stores the generated documentation as YAML files. This code uses RDoc to parse the YAML files:

   1  require 'yaml'
   2  require 'find'
   3  require "rdoc/ri/ri_driver"
   4  
   5  dirs = RI::Paths::PATH
   6  dirs.each do |dir|
   7    Find.find(dir) do |fn|
   8      next unless File.file?(fn)
   9      doc = YAML.load(File.read(fn))
  10      next unless doc.respond_to?(:comment)
  11      next unless doc.comment
  12      
  13      # Print name of object
  14      puts doc.full_name
  15      
  16      # Print the body: RDoc comments, but only partial...
  17      puts doc.comment.map{|f| f.body if f.respond_to?(:body)}.join("\n")
  18    end
  19  end

Originally from the article Fun with Ferret.

Tagged rdoc, ri, documentation, ruby

How to parse Ruby source code documentation with RDoc and a custom RDoc generator

Ruby posted 11 months ago by christian

This is a skeleton for an RDoc generator that extends the existing HtmlGenerator. This means we get the same documentation as seen at, for example, http://api.rubyonrails.org/; with links and HTML formatted documentation.

It can be used for doing whatever you would like and can imagine doing with RDoc documentation. Currently it prints out the files, modules, classes and methods found in the processesed files.

To use it, create a new file named custom_generator.rb in the Ruby installation and the subfolder /rdoc/generators. Then put the following code in the file:

   1  require 'rdoc/generators/html_generator'
   2  
   3  module Generators
   4  
   5    class HTMLGenerator
   6    
   7      # We don't need a template
   8      def load_html_template
   9      end
  10  
  11      def generate(toplevels)             
  12        @toplevels  = toplevels
  13        @files      = []
  14        @classes    = []
  15  
  16        build_indices
  17        
  18        puts "===================="
  19        puts "Files"
  20        puts "===================="
  21        
  22        @files.each do |item|
  23          puts item.name
  24          #values = file.value_hash
  25          #puts item.description
  26        end
  27        
  28        puts "===================="
  29        puts "Modules and classes"
  30        puts "===================="           
  31        
  32        @classes.each do |item|
  33          puts item.name
  34          #values = file.value_hash
  35          #puts item.description
  36        end
  37        
  38        puts "===================="
  39        puts "Methods"
  40        puts "===================="      
  41        
  42        HtmlMethod.all_methods.each do |item|
  43          puts item.name
  44        end
  45      end
  46    end
  47    
  48    class HtmlFile
  49      # Add a description method, after all HtmlMethod has it...
  50      def description
  51        value_hash if @values.size == 0      
  52        @values["description"]
  53      end
  54    end
  55    class HtmlClass
  56      # Add a description method, after all HtmlMethod has it...
  57      def description
  58        value_hash if @values.size == 0      
  59        @values["description"]
  60      end
  61    end
  62    
  63    class CUSTOMGenerator < HTMLGenerator
  64    end
  65  
  66  end

Then run the custom generator by using the fmt parameter:

   1  rdoc --fmt custom lib/base64.rb lib/pp.rb

You can also control RDoc programatically, with the following code:

   1  #!/usr/bin/env ruby
   2  require 'rdoc/rdoc'
   3  
   4  `rm -rf doc`
   5  
   6  begin
   7    r = RDoc::RDoc.new
   8    r.document(['--inline-source', '--fmt', 'custom'] + ARGV)
   9  rescue RDoc::RDocError => e
  10    $stderr.puts e.message
  11    exit(1)
  12  end

Tagged ruby, rdoc, generator, documentation

How to create a custom RDoc generator

Ruby posted 11 months ago by christian

This is a work in progress… Heavily influenced by RAnnotate. Copy this to rdoc/generators:

   1  module Generators
   2  
   3    class CUSTOMGenerator                 
   4  
   5      TYPE        = {:file => 1, :class => 2, :module => 3 }
   6      VISIBILITY  = {:public => 1, :private => 2, :protected => 3 }
   7          
   8      def CUSTOMGenerator.for(options)
   9        new(options)
  10      end
  11          
  12      def initialize(options) #:not-new:
  13        @options = options
  14        
  15        # set up a hash to keep track of all the classes/modules we have processed
  16        @already_processed = {}
  17        
  18        # set up a hash to keep track of all of the objects to be output
  19        @output = {
  20          :files => [], 
  21          :classes => [], 
  22          :modules => [], 
  23          :attributes => [], 
  24          :methods => [], 
  25          :aliases => [], 
  26          :constants => [], 
  27          :requires => [], 
  28          :includes => []
  29        }
  30      end
  31  
  32      # Rdoc passes in TopLevel objects from the code_objects.rb tree (all files)
  33      def generate(files)                             
  34        # Each object passed in is a file, process it
  35        files.each { |file| process_file(file) }
  36      end
  37  
  38      private
  39  
  40      # process a file from the code_object.rb tree
  41      def process_file(file)
  42        @output[:files].push(file)
  43  
  44        puts "#{file.comment}"
  45            
  46        # Process all of the objects that this file contains
  47        file.method_list.each { |child| process_method(child) }
  48        file.aliases.each { |child| process_alias(child) }
  49        file.constants.each { |child| process_constant(child) }
  50        file.requires.each { |child| process_require(child) }
  51        file.includes.each { |child| process_include(child) }
  52        file.attributes.each { |child| process_attribute(child) }   
  53      
  54        # Recursively process contained subclasses and modules 
  55        file.each_classmodule do |child| 
  56            process_class_or_module(child)      
  57        end   
  58      end
  59      
  60      # Process classes and modiles   
  61      def process_class_or_module(obj)
  62        obj.is_module? ? type = :modules : type = :classes
  63   
  64        # One important note about the code_objects.rb structure. A class or module
  65        # definition can be spread a cross many files in Ruby so code_objects.rb handles
  66        # this by keeping only *one* reference to each class or module that has a definition
  67        # at the root level of a file (ie. not contained in another class or module).
  68        # This means that when we are processing files we may run into the same class/module
  69        # twice. So we need to keep track of what classes/modules we have
  70        # already seen and make sure we don't create two INSERT statements for the same
  71        # object.
  72        if(!@already_processed.has_key?(obj.full_name)) then      
  73          @output[type].push(obj)
  74          @already_processed[obj.full_name] = true
  75            
  76          # Process all of the objects that this class or module contains
  77          obj.method_list.each { |child| process_method(child) }
  78          obj.aliases.each { |child| process_alias(child) }
  79          obj.constants.each { |child| process_constant(child) }
  80          obj.requires.each { |child| process_require(child) }
  81          obj.includes.each { |child| process_include(child) }
  82          obj.attributes.each { |child| process_attribute(child) }   
  83        end
  84        
  85        id = @already_processed[obj.full_name]
  86        # Recursively process contained subclasses and modules 
  87        obj.each_classmodule do |child| 
  88        	process_class_or_module(child) 
  89        end
  90      end       
  91      
  92      def process_method(obj)  
  93        obj.source_code = get_source_code(obj)
  94         puts "#{obj.name}#{obj.param_seq}"
  95         puts "#{obj.source_code}" # Source code, unformatted
  96  
  97         puts "====================================="
  98  
  99        @output[:methods].push(obj)                                
 100      end
 101      
 102      def process_alias(obj)
 103        @output[:aliases].push(obj)  
 104      end
 105      
 106      def process_constant(obj)
 107        @output[:constants].push(obj)    
 108      end
 109      
 110      def process_attribute(obj)
 111        @output[:attributes].push(obj)     
 112      end
 113      
 114      def process_require(obj)
 115        @output[:requires].push(obj) 
 116      end
 117      
 118      def process_include(obj) 
 119        @output[:includes].push(obj)     
 120      end   
 121      
 122                 
 123      # get the source code
 124      def get_source_code(method)
 125        src = ""
 126  	  if(ts = method.token_stream)    
 127  	    ts.each do |t|
 128  	    next unless t    			
 129  	      src << t.text
 130  	    end
 131        end
 132        return src
 133      end
 134           
 135    end
 136  
 137  
 138     # dynamically add a source code attribute to the base oject of code_objects.rb
 139     class RDoc::AnyMethod
 140       attr_accessor :source_code	  
 141     end
 142  
 143  end
 144  
 145  

Tagged rdoc, ruby, generator