Meeting Notes

From SF Data Wiki

Sep 17, 2009

Agenda

1. Next meetup event
2. Review use cases
3. Discuss status of processors

Sep 12, 2009

CivicDB September Meetup - Full day on Sat

Agenda

1. Who's who?
2. Jamie Taylor: a quick overview of Freebase and how it might be useful
3. Breakout

    Data source track

1. Migrating to Oregon State for servers
2. Discussion on sponsorship of CivicDB
3. What's not yet in DataSF.org and why?
4. How to get more data sources?
5. Next steps: DataOakland vs DataCA
6. How to reach out to other cities?

    Software track - How to convert government data (ex. DataSF.org) to computer readable format?

1. Design
   • Example data
   • Use cases
   • Metadata
   • Neutral data format (ex. triple) vs "Good data/Exception" type processing
2. Code

Sep 10, 2009

Action Items

1. Matt to test CSV-->XML processor with additional datasets
2. Mano to continue investigating geo processors (plane transformations and geocoding)
3. Csaba and Tom to develop additional processors

Agenda

1. Review Metadata
2. Status on processor testing
3. Status on new processors
4. Discuss 9/12 Meetup event

Sep 3, 2009

Agenda

1. Review prototype status
2. Plan for CivicDB meetup
3. Migrating to Oregon State for servers

Action Items

• Josh to design and implement prototype of storing metadata associated with datasets
• Matt to run csv-->xml processor through additional datasets
• Tom and Csaba to create additional processors
• Someone to design geo processor (Mano?)

Aug 13, 2009

Agenda

1. Infrastructure issues
2. Status on prototyping
3. Discussion on sponsorship of CivicDB

Action Items

1. Tom to refine #3 and submit 5

2. Tom initiate conversation on mailing list of different technologies behind prototype

3. SF to setup Mailing list on for CivicDB 8/20

4. SF to setup private wiki on CivicDB.net

5. SF to continue discussing with Socrata & MS on getting code for their products

6. Josh to create use cases on data input 

July 30, 2009

Agenda

• Software license
• Prototype status
• Mailman status
• Github repository

Action Items

• Software license
  • Summary of Pros/Cons for GPL and BSD/Apache to apply to final project
  • Apply BSD license to prototype
• Prototype status
  • Tom to investigate technologies for each part of architecture
  • Jared, Terry to investigate technologies for data cleaning
• Mailman status
  • Matt to set up new mailman for civicdb domain 
  • Matt and Josh to co-ordinate on  moving archives from datasf mailing list to civicdb mailman
• Github repository
  • Jay to set up CivicDB user on Github
  • Everyone to send Github usernames to Jay
  • Tom to introduce Jay to Github folks
  • Jay to trial Git inside SF DoT

July 23, 2009

Agenda

• Review prior action items
  • Use case #3 needs to be updated by Csaba
  • Use case #5 needs to be created by Tom
• Outreach updates
  • Selected for Gov 2.0 Expo (nice work Kelly and Tom!)
  • Met with Craig Newmark
  • Publicity page created
• CivicDB website/logo
  • looking for graphic designers interested in helping with logo or WP theme
  • draft version created - more versions coming
• Server status
  • working with 3Tera cloud provider; expecting availability Fri/Mon.
  • noon Mon cutoff then plan b (EC2)
• Review final architecture
• Discuss prototype design
  • Update to processing module to indicate flexibility in language; sample code will be provided
  • Hadoop seems to be overkill; need to be assessed as we prototype
  • Mano suggested:
    • Automated registry creation on raw storage (during processing or part of storage platform)
    • GeoServer or Geo Network may help
    • Use sitemap for better site usability and SEO
• Software license
  • Quick selection of license is needed
  • Agreed that liberal license like Apache, CC0 is desired

Action Items

• Select software license - All
• Provide prototype server by 7/27 - Jay
• Update use case# 3 - Csaba
  
• Create use case #5 - Tom
• Create several website themes- Jay
  

July 16, 2009

Agenda

• Review previous action items
• Project name
• Review project milestones
• Begin selecting technology candidates for arch

Action Items

1. Csaba to finalize arch with Tom
2. Matt to document initial technology stack
3. Jay to acquire server in DMZ
4. Jay to create CivicDB website
5. Renuka to update wiki with CivicDB name
6. Please update our project team list if you’re interested
   

July 9, 2009

Agenda

• Introductions
• Outreach efforts
• Gov2.0 proposal presentation
• Project Name
• Roadmap
• Review action items status
• Review technical architecture
• Review data consumer requirements
• Summer of Gov - Tuesday, July 14, 2009 from 6:00 PM - 9:00 PM (PT)

Action Items

1. Csaba to update use case #3
   1. Add ACL to component 2
   2. Update processing to reflect internal and external components
2. Csaba to add use case #4 – data on demand (ie not batch oriented)
3. Josh to inquire about open geo standards
4. Jay to send poll on project name
5. Jay to draft roadmap/milestones
6. Jay to solicit input on data consumer reqs from other groups
7. Jay to clean up wiki to reflect new pages (e.g. arch)

July 2nd, 2009

Agenda

• New member introductions
• Outreach efforts to other cities
• Project Phases
• Project Requirements
• Project Success Criteria
• Review information architecture design
• Standard data schemas – possible?
• Process for selecting underlying technologies

Action Items

Outreach

1. Kelly to contact Vancouver
   1. In touch with David Eaves (non-city employee) and Kevin Bowers (Manager, Technology Planning for City of Vancouver). Scheduled conference call for Tuesday, July 7 at 3pm. 
2. Josh to continue efforts with Portland- contact with OSU with Portland State Univeristy (Deborah Bryant at OSCON)
   
3. Alissa to reach out to NYC
4. Jay to continue efforts with Sunlight and NYC through Craig Newmark
5. Kelly and Tom to submit proposal for Gov 2.0 proposal by 7/7
6. Jay to seek sponsors for July 14th SF Gov 2.0 event at 111 Minna
7. Jay to reach out to Adriel for Gov 2.0 radio appearance

Technical

1. Csaba to document hybrid data architecture in wiki (A2 & A3)

Meeting Notes

• Outreach
  • Jay is blogging at OpenSF. Kelly is also blogging at (the admittedly less interesting and less updated) Innovation City.
  • Scheduling conference calls with Sunlight Foundation and MAPlight in the coming week.
  • Reaching out to: Boston, Vancouver, NY, Portland and Apache Foundation.
  • Will be having a kickoff event on July 14, 2009 at 111 Minna (details to come).

• Project Phases

1. Portal Approach
   1. Less sexy and less cool, but easiest to put together in the short term.
   2. Deadline of August 1.
   3. Most important thing is to make the data available. Fine tuning comes later.
   4. Simultaneous solicitation of requests for certain data sets.
   5. Will be worked on concurrently with phase 2.
2. Better Approach (Pre-Processing Data/Decentralized Converters)
   1. More time consuming to construct, but ultimately more efficient and useful.
   2. Framework for people to share converted data?

• Requirements
  • See updated project requirements
  • Unresolved: Where/How does the ability for the public to submit data corrections fit into our requirements/measurements for success?

• Success
  • Where/How do operational costs and cost-effectiveness fit into our vision of success?

• Miscellaneous
  • Discussed the current controversy with SFMTA/Next Bus Information Systems. Next Bus (which handles the GPS tracking of Muni buses) has issued cease and desist letters to third party application developers who use their data. SFMTA has publicly disagreed with their decision. Reinforces need for open data and reformed T & C boilerplates.

• Long Term Vision/ Action Items
  • Internal acquisition of data.
  • Outside architecture.
  • Policy/Publicity.

June 25th, 2009

• Introductions
• Project goals and background
• Information architecture: (1) Portal - URLs to dataset location; (2) Decentralized Converters, Standard Data Format; (3) Pre-processing Data Approach 
  
• Decision to be made by simple majority vote on information architecture